I understand you can use Inject anything to embed simple html files. But I was wondering once the module has been added if I update the html file will this update propogate back to the page where it has been embedded?

Or do I have to re-embed the module with the html file to apply the update?

No, the html will be read each time unless you have caching enabled on that module. You don't have to re-embed the module each time you update the html.