Running out of memory - Lightning node

Why is that…

Here’s a copy of the error message:

12910 texture patches.
Running global seam leveling:
Create matrices for optimization…
done.
Lhs dimensionality: 220076 x 220076
Calculating adjustments:
Color channel 0: CG took 119 iterations. Residual is 8.8939e-05
Color channel 2: CG took 116 iterations. Residual is 9.93064e-05
Color channel 1: CG took 119 iterations. Residual is 8.9781e-05
Took 0.997 seconds
Adjusting texture patches 100%… done. (Took 11.519s)
Running local seam leveling:
Blending texture patches 100%… done. (Took 175.265s)

Generating texture atlases:
Sorting texture patches…
done.
Killed
===== Dumping Info for Geeks (developers need this to fix bugs) =====
Child returned 137
===== Done, human-readable information to follow… =====

[ERROR] Whoops! You ran out of memory! Add more RAM to your computer, if you’re using docker configure it to use more memory, for WSL2 make use of .wslconfig (Advanced settings configuration in WSL | Microsoft Learn), resize your images, lower the quality settings or process the images using a cloud provider (e.g. https://webodm.net).
Traceback (most recent call last):
File “/code/stages/odm_app.py”, line 83, in execute
self.first_stage.run()
File “/code/opendm/types.py”, line 338, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 338, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 338, in run
self.next_stage.run(outputs)
[Previous line repeated 4 more times]
File “/code/opendm/types.py”, line 319, in run
self.process(self.args, outputs)
File “/code/stages/mvstex.py”, line 108, in process
system.run('“{bin}” “{nvm_file}” “{model}” “{out_dir}” ’
File “/code/opendm/system.py”, line 90, in run
raise SubprocessException(“Child returned {}”.format(retcode), retcode)
opendm.system.SubprocessException: Child returned 137
100 - done.

It does indeed seem like it’s running out of memory (which is strange, we allocate plenty of memory for processing ~200 images). Is there something different/unique about this dataset? Is the GCP formatted correctly?

1 Like

No nothing special. Smart oblique mission with a M300 + P1 camera.

EPSG:3006
592879.8005 6590690.6128 12.8335 5902.934012738853 560.4907006369426 GCP2_4.jpg P2
592879.8005 6590690.6128 12.8335 1938.93949044586 2971.911847133758 GCP2_1.jpg P2
592879.8005 6590690.6128 12.8335 3816.686624203822 4253.15872611465 GCP2_2.jpg P2
592879.8005 6590690.6128 12.8335 3722.4002547770697 1191.6971974522291 GCP2_3.jpg P2
592863.4817 6590683.2751 17.4688 4706.8285350318465 2099.971592356688 GCP5_4.jpg P5
592863.4817 6590683.2751 17.4688 2919.6917197452226 1639.7177070063694 GCP5_1.jpg P5
592863.4817 6590683.2751 17.4688 1857.5238216560508 2001.0286624203823 GCP5_2.jpg P5
592863.4817 6590683.2751 17.4688 1311.8923566878982 3364.0853503184717 GCP5_3.jpg P5
592859.5402 6590686.6566 17.7381 3716.333248407643 3394.8894267515925 GCP6_1.jpg P6
592859.5402 6590686.6566 17.7381 1363.9727388535032 3074.9250955414013 GCP6_2.jpg P6
592859.5402 6590686.6566 17.7381 7095.930191082803 1987.3431847133759 GCP6_3.jpg P6
592838.157 6590676.0588 13.3073 1649.9814012738852 3252.9941401273886 GCP8_3.jpg P8
592838.157 6590676.0588 13.3073 3559.911337579618 1199.6890445859872 GCP8_1.jpg P8
592838.157 6590676.0588 13.3073 3526.4649681528663 2584.310828025478 GCP8_2.jpg P8
592865.182 6590676.0524 14.2732 4773.419639448568 4509.467826086957 GCP10_1.jpg P10
592865.182 6590676.0524 14.2732 690.375923566879 4087.4425477707005 GCP10_3.jpg P10
592865.182 6590676.0524 14.2732 1287.0985987261147 4774.208917197452 GCP10_2.jpg P10
592852.9668 6590674.5908 13.4765 4684.488483563096 4012.7019300106044 GCP9_2.jpg P9
592852.9668 6590674.5908 13.4765 5963.60619300106 2449.8468716861084 GCP9_1.jpg P9
592852.9668 6590674.5908 13.4765 3364.055796178344 2277.992356687898 GCP9_3.jpg P9
592844.0078 6590669.1268 13.1907 6513.614591728526 575.8053446447508 GCP7_2.jpg P7
592844.0078 6590669.1268 13.1907 6698.675715800637 3452.7630116648993 GCP7_1.jpg P7
592844.0078 6590669.1268 13.1907 6563.3722292993625 1331.1429299363058 GCP7_3.jpg P7
592872.7167 6590688.3715 13.8648 4327.356606574761 2189.8881018027573 GCP4_2.jpg P4
592872.7167 6590688.3715 13.8648 2274.642799575822 3109.2584517497353 GCP4_1.jpg P4
592872.7167 6590688.3715 13.8648 1317.4974522292994 4469.367133757962 GCP4_3.jpg P4
592868.5495 6590691.5649 13.761 5965.2106044538705 4031.471983032874 GCP3_2.jpg P3
592868.5495 6590691.5649 13.761 2510.2338918345704 4207.301293743372 GCP3_1.jpg P3
592868.5495 6590691.5649 13.761 1481.6015286624204 4985.941401273885 GCP3_3.jpg P3
592876.4921 6590698.518 11.6017 1755.738366914104 1079.0291410392365 GCP1_1.jpg P1
592876.4921 6590698.518 11.6017 2179.143439490446 4665.472101910828 GCP1_3.jpg P1
592876.4921 6590698.518 11.6017 2581.816050955414 2851.275668789809 GCP1_2.jpg P1

I would try to remove some GCPs (you have 10 of them). Perhaps leave P7, P1 and P5. This will reduce the possibility of mistags. If it works, add 2 more for additional accuracy. There’s probably little benefit in having 10 GCPs (for such a small area, too).

1 Like

Ok - will try.

I did 10 GCPs because we are testing out the difference between RTK vs GCP missions with the new (for us) M300 drone.

I did resize the images to 2048 px and the task went okey - but the quality was not good at all. I will try to remove a couple of GCPs and try again.

1 Like

Resizing the images helped, task completed.

Leaving 3 of the 10 GCPs out did not help - not enough memory.

Both task via the lightning node.

1 Like

How large are these images?

1 Like

The p1 camera is 45Mp full-frame so the images is 8192x5460 [16mb]

2 Likes

I’m afraid 200 images at 45MP would be prohibitive with that much memory without downsampling.

Cool sensor though!

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.