Orthophoto fails when GPS is added to EXIF

I am attempting to create an orthomosaic using images pulled from a video. The camera used during the flight was a GoPro and so the resulting images are not georeferenced. However, the drone did collect GPS data, so when pulling the right frames from the video, I interpolated the GPS data I had and manually added it to the JPEG images using piexif.

Although the GPS points appear correct when I load the image set into QGIS, when I try to create an orthophoto using the georeferenced images I get nonsensical results. The mosaic using the unreferenced imagery isn’t perfect (the video was shaky), but it is worlds apart from the results I get with the referenced images.

The images I used and their orthophotos can be seen here: Google Drive

I’m sorry, but I no longer have the task output available. I can rerun the jobs if that would be helpful. I used only default parameters.

Am I missing a piece of EXIF data needed beyond Lat Long and Altitude?