Will driving a car with no alternator belt cause it to overheat ('97 plymouth neon)?

No your car will not overheat. If the battery is powering the car albeit in a somewhat diminished manner it is powering everything. That would include the radiator and fans as well. It won't have the chance to overheat because once the radiator and fans stop working the rest of your car components such as the engine transmission etc would stop working. You cant overheat the engine if its not running in the first place.