Running out of memory - Lightning node

Why is that…

Here’s a copy of the error message:

12910 texture patches.
Running global seam leveling:
Create matrices for optimization…
done.
Lhs dimensionality: 220076 x 220076
Calculating adjustments:
Color channel 0: CG took 119 iterations. Residual is 8.8939e-05
Color channel 2: CG took 116 iterations. Residual is 9.93064e-05
Color channel 1: CG took 119 iterations. Residual is 8.9781e-05
Took 0.997 seconds
Adjusting texture patches 100%… done. (Took 11.519s)
Running local seam leveling:
Blending texture patches 100%… done. (Took 175.265s)

Generating texture atlases:
Sorting texture patches…
done.
Killed
===== Dumping Info for Geeks (developers need this to fix bugs) =====
Child returned 137
===== Done, human-readable information to follow… =====

[ERROR] Whoops! You ran out of memory! Add more RAM to your computer, if you’re using docker configure it to use more memory, for WSL2 make use of .wslconfig (Manage Linux Distributions | Microsoft Docs), resize your images, lower the quality settings or process the images using a cloud provider (e.g. https://webodm.net).
Traceback (most recent call last):
File “/code/stages/odm_app.py”, line 83, in execute
self.first_stage.run()
File “/code/opendm/types.py”, line 338, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 338, in run
self.next_stage.run(outputs)
File “/code/opendm/types.py”, line 338, in run
self.next_stage.run(outputs)
[Previous line repeated 4 more times]
File “/code/opendm/types.py”, line 319, in run
self.process(self.args, outputs)
File “/code/stages/mvstex.py”, line 108, in process
system.run(’"{bin}" “{nvm_file}” “{model}” “{out_dir}” ’
File “/code/opendm/system.py”, line 90, in run
raise SubprocessException(“Child returned {}”.format(retcode), retcode)
opendm.system.SubprocessException: Child returned 137
100 - done.

chrome_4EDgAqEGRy

It does indeed seem like it’s running out of memory (which is strange, we allocate plenty of memory for processing ~200 images). Is there something different/unique about this dataset? Is the GCP formatted correctly?

1 Like

No nothing special. Smart oblique mission with a M300 + P1 camera.

EPSG:3006
592879.8005 6590690.6128 12.8335 5902.934012738853 560.4907006369426 GCP2_4.jpg P2
592879.8005 6590690.6128 12.8335 1938.93949044586 2971.911847133758 GCP2_1.jpg P2
592879.8005 6590690.6128 12.8335 3816.686624203822 4253.15872611465 GCP2_2.jpg P2
592879.8005 6590690.6128 12.8335 3722.4002547770697 1191.6971974522291 GCP2_3.jpg P2
592863.4817 6590683.2751 17.4688 4706.8285350318465 2099.971592356688 GCP5_4.jpg P5
592863.4817 6590683.2751 17.4688 2919.6917197452226 1639.7177070063694 GCP5_1.jpg P5
592863.4817 6590683.2751 17.4688 1857.5238216560508 2001.0286624203823 GCP5_2.jpg P5
592863.4817 6590683.2751 17.4688 1311.8923566878982 3364.0853503184717 GCP5_3.jpg P5
592859.5402 6590686.6566 17.7381 3716.333248407643 3394.8894267515925 GCP6_1.jpg P6
592859.5402 6590686.6566 17.7381 1363.9727388535032 3074.9250955414013 GCP6_2.jpg P6
592859.5402 6590686.6566 17.7381 7095.930191082803 1987.3431847133759 GCP6_3.jpg P6
592838.157 6590676.0588 13.3073 1649.9814012738852 3252.9941401273886 GCP8_3.jpg P8
592838.157 6590676.0588 13.3073 3559.911337579618 1199.6890445859872 GCP8_1.jpg P8
592838.157 6590676.0588 13.3073 3526.4649681528663 2584.310828025478 GCP8_2.jpg P8
592865.182 6590676.0524 14.2732 4773.419639448568 4509.467826086957 GCP10_1.jpg P10
592865.182 6590676.0524 14.2732 690.375923566879 4087.4425477707005 GCP10_3.jpg P10
592865.182 6590676.0524 14.2732 1287.0985987261147 4774.208917197452 GCP10_2.jpg P10
592852.9668 6590674.5908 13.4765 4684.488483563096 4012.7019300106044 GCP9_2.jpg P9
592852.9668 6590674.5908 13.4765 5963.60619300106 2449.8468716861084 GCP9_1.jpg P9
592852.9668 6590674.5908 13.4765 3364.055796178344 2277.992356687898 GCP9_3.jpg P9
592844.0078 6590669.1268 13.1907 6513.614591728526 575.8053446447508 GCP7_2.jpg P7
592844.0078 6590669.1268 13.1907 6698.675715800637 3452.7630116648993 GCP7_1.jpg P7
592844.0078 6590669.1268 13.1907 6563.3722292993625 1331.1429299363058 GCP7_3.jpg P7
592872.7167 6590688.3715 13.8648 4327.356606574761 2189.8881018027573 GCP4_2.jpg P4
592872.7167 6590688.3715 13.8648 2274.642799575822 3109.2584517497353 GCP4_1.jpg P4
592872.7167 6590688.3715 13.8648 1317.4974522292994 4469.367133757962 GCP4_3.jpg P4
592868.5495 6590691.5649 13.761 5965.2106044538705 4031.471983032874 GCP3_2.jpg P3
592868.5495 6590691.5649 13.761 2510.2338918345704 4207.301293743372 GCP3_1.jpg P3
592868.5495 6590691.5649 13.761 1481.6015286624204 4985.941401273885 GCP3_3.jpg P3
592876.4921 6590698.518 11.6017 1755.738366914104 1079.0291410392365 GCP1_1.jpg P1
592876.4921 6590698.518 11.6017 2179.143439490446 4665.472101910828 GCP1_3.jpg P1
592876.4921 6590698.518 11.6017 2581.816050955414 2851.275668789809 GCP1_2.jpg P1

I would try to remove some GCPs (you have 10 of them). Perhaps leave P7, P1 and P5. This will reduce the possibility of mistags. If it works, add 2 more for additional accuracy. There’s probably little benefit in having 10 GCPs (for such a small area, too).

image

1 Like

Ok - will try.

I did 10 GCPs because we are testing out the difference between RTK vs GCP missions with the new (for us) M300 drone.

I did resize the images to 2048 px and the task went okey - but the quality was not good at all. I will try to remove a couple of GCPs and try again.

1 Like

Resizing the images helped, task completed.

Leaving 3 of the 10 GCPs out did not help - not enough memory.

Both task via the lightning node.

1 Like

How large are these images?

1 Like

The p1 camera is 45Mp full-frame so the images is 8192x5460 [16mb]

2 Likes