Thermal orthomosaic. Experiences and suggestions?

Hi,

I have been using ODM for RGB othomosaics for the last couple of years and it is nice to see I will be able to use it for agricultural purposes in the short term. Nevertheless, I also do a lot of thermal inspection and cannot find an affordable software to create my own orhtomosaics.

I am aware from some topics that some people have succeeded on creating thermal orthos using ODM. Nevertheless, I wanted to ask abut your experience. I own a DJI Zenmuse XT and I wanted to know:

  • Can you somehow process data with ODM so you can later keep and explore pixels temperature? If so, what is your workflow?
  • How much overlap and flight altitude is better for ODM thermal processing.
  • If absolute temperature is not maintained… how do you later inspect your data? Do you just go for the original image in the problematic point and extract real absolute data from there?

Finally, are there any plans to include native thermal processing in ODM? What is it so difficult to find good thermal orthomosaic software out there? In the past I have been working with PrecisionMapper but they dropped thermal suppond and now rely on third party companies to analyze our data. And it does not seem to be a straight process with lots of problems emerging from time to time.

Thanks in advance,

Jose

1 Like

Hi Jose,

There are all great questions. There have been some thermal datasets shared as part of the multispectral work, and I think we have an interest, at least in the mid-term, of including thermal processing under the umbrella. It has similar questions of: quantitative and often calibrated data; often stored in integer form with some function to transform to calibrated units; and a bunch of per-manufacturer data storage standards. The last point, given the dominance of FLIR might be less true.

If you are willing to share a dataset with @pierotofy, it will help grow the tool, though I don’t know timeline, as he currently doesn’t have a mandate to include thermal that I know of. :slight_smile:

Hi!,

Thanks for your response. I will be willing to provide as many thermal image sets as you want. Both Tiff , JPEGs and R-JPEG. Not only that but I would be delighted to help you coding the needed algorithms if:

  • You provide me some help to start getting familiar with ODM coding internals.
  • You know of some source of documentation regarding building thermal orthomosaics that deals with specific issues of thermal imaging.

I have been a computer scientist for almost 20 years and am also working now in the drones world thanks to my background on image processing (although a bit outdated in recent years :)).

Cheers,

Jose

1 Like

Hey @Jose_Roman_Bilbao_Ca :hand: we’d welcome any help!

For getting started with ODM coding, see https://github.com/OpenDroneMap/ODM#developers. We don’t have a whole lot of documentation for internals, but ask if you have questions!

I don’t have any specific resources to point to for thermal cases, but if you find some please post them in our research papers area on this forum :+1:

1 Like

I have done some Thermal Orthomosaic imaging. Overlap is critical. I used 80% in both dimensions. If you want absolute radiometry, you need some standards in the scene; a wading pool of water with a bimetalic temperature probe is great. I found I didn’t need absolute temperature as I was doing flat roof inspection and the delta T was adequate. Once I was sure of a bad portion of roof in the scene, I used the semi-automatic classification plug in to classify the other leak locations. The particular roof had been “ground-truthed” by a local roofing company. After all that, the leaking roof assessment was about 100% good.

2 Likes

I’ve also tinkered with thermal orthomosaics. from my pov mapping ortho RGB outputs to thermal values is the key. My particular instrument (ANAFIthermal) has low spatial resolution for the thermal sensor, but great greyscale resolution (21MP) - the two are combined for viewing; and ODM uses that combined RGB (thermal + greyscale @ 21MP) to do modelling.

However, it’s a FLIR sensor and exiftool reveals that the 640x480 thermal image is extractable, and all the thermal metrics are present in exif… sooo… perhaps examining exif for thermal data and working out some mapping to the combined RGB image could be a thing?

Parrot stores a LOT in exif, which is great - I can only presume other instruments do the same?

1 Like

Hi, thanks for your post. I didnt mean how to do thermal inspections as I already work on it profesionally :slight_smile: , but on the orthomosaic part as this is the part that I need to rely on external companies to accomplish. I am looking to implement this functionality on ODM, but I need some algorithmic details to improve thermal processing (it seems to be a bit special)

The Anafi is a great tool for on-site inspection or “crude” inspections where not too much detail is needed. In the solar inspection workflow, for example, resolution needs to be high and thermal precision needs to be too. The Anafi has too a low resolution to successfully and consistently create thermal orthomosaics. By the way, how is that that you say that the 640x480 is there? It is a 160 x 120 sensor.

I can weigh in here. The anafi upscales the image from it’s lower sensor dimension. I think this is due to many popular processing softwares more easily integrating a standard thermal resolution (640x480) even though the sensor only captures 160x120.

Also they can market the camera as able to produce 640x480 images…

1 Like

My use of the ANAFI is just an example of what can already happen in ODM, and maybe point at where things might go processing wise.

Here is a sample image. Its a merged greyscale image from the 21MP camera, covering the thermal sensor FOV with ‘colour’ applied to whichever 21MP pixels exist in the FOV.

P0741911

Running ODM on this, it just reads as if its an RGB image - no accounting for whatever temp values are represented. That’s why I mentioned getting some clear ideas about what an RGB value from an output image means in the resulting orthophoto, like this one:

odm_orthophoto

Right now there’s no way of linking what the colours in that result mean to a thermal value - even using an in-scene target as @konradkern suggested, there’s no way to map extrema. For this instrument, examining the image metadata and doing some maths will help sort that out.

…even for pure thermal imagers, you’ll still need to know what means 0 and what means 256 in an rgb output - so calling on the exif data in any new thermal capability will help. I bet there are clues in recent multispectral work by @pierotofy!

What tags do your sensors apply? do other sensors (eg workswell) still apply FLIR tags? (https://exiftool.org/TagNames/FLIR.html)

Love the idea of the ANAFI dual camera :-). You are right you cannot extract temp info. I suppose, in this case where temperature information is just disregarded and just RGB information is applied, something like Thermal “ground control points” should be applied somehow. Nevertheless, the logistics to provide two bodies with two known different temperatures are so complex that I don’t think it is feasible.

That is a good (although unwise) commercial strategy. If you are serious on thermal imaging (radiometric inspections), anything less than 640 is not very useful as you will lose a lot of information. And interpolation here is a very bad idea…

Agreed. I am happy that Parrot rubbed shoulders with DJI and released a cheap consumer thermal option and look forward to future iterations. Any competition in the consumer market with DJI is good (not anti-china, just anti-monopoly). But the resolution limits you to more active applications, like SAR or thermal scouting as opposed to analysis.

Excited to see what the Autel Evo II thermal has for us! I’ve tried to get official specs from them, but all I can find is that it’s a FLIR Boson sensor that they advertise as producing 640 resolution photos. Whether that is up-scaled or is the official Boson 640 model is to be determined upon release in March!

1 Like

Interpolation is the bread, butter, cheese and jam of digital imaging :wink: - so long as we understand what is being interpolated all is well; and further - we can always alter sample size by flying closer to or further away from things. In my experience SAR is not a fantastic use case for low resolution sensors, and I’m also curious about the Autel Evo II thermal - but thats all an aside…

…back to the topic of thermal orthos - I have a strong hunch we can get to a useful place using the radiometric data stored in FLIR EXIF tags. If anyone would like to fund a project, I have some time I can commit :wink:

1 Like