If I leave Japan and obtain an international driving permit overseas, can I drive with that permit after returning to Japan?

It depends how long you stay abroad/outside Japan. In case you reenter Japan in less than three months after leaving the country, you shall not drive with your international driving permit even if it is a new one you have obtained during such a stay outside Japan. On the other hand, in case you are a temporary visitor or a resident in Japan, who is not registered on the Basic Resident Register (Jumin Kihon Daicho), you may drive with your international permit regardless of the period of your absence in Japan prior to the entry.
Valid International Driving Permit in Japan