LiveODM limited storage

Hello everyone

I’m using WebODM via docker for a while now with no issues, been having and awesome time with it.
But now I have to set up a LiveODM USB stick, and im having some issues.

I have a 64Gb stick, flashed the iso with no problems, but im working with a large data set (820 images with 6,5Gb). WebODM will upload the files, but always get stuck when uploading images to the node, after a while nothing works on webODM until i reboot the system.
I believe the problem is due to limited storage webODM is allowed to use… When checkin the Diagnostic tab, it shows only 7,8Gb of available space. Also, on Lubuntu, when checking the home folder, it shows only 7,8Gb total storage space, with 0 bytes free.

So, is there any way to enlarge the available space for webODM to work? Did I do something wrong while setting up the .iso or flashing the image?

Appreciate any help!

1 Like

Welcome!

LiveUSBs typically only have a limited amount of space available in the loop FS for persistent LiveUSBs. Depending upon the tool used to flash the image and the underlying filesystem, the maximum is somewhere in the neighborhood of 4GB.

This article helps explain it, and a work-around:
https://www.pendrivelinux.com/how-to-create-a-larger-casper-rw-loop-file/

You’d ideally want to do a native/direct-to-hardware install of the LiveODM image onto the stick, IF you want to process/store directly on the stick.

I’ve spent a long time trying to work up a reliable method to do so locally, but I have had limited success with making a bootable “native” LiveODM stick on a 64GB USB drive I had. The process is pretty fraught, however.

Thanks for the response!
I don’t understand a lot about bootable drives or persistent file systems, so maybe im not understanding it correctly, but my problem isnt really with persistency… The problem is webodm only gets 7.8Gb of storage to work with, and thats not enough for larger datasets… This process you posted will allow webodm to use more space from the drive?

1 Like

So, LiveUSB should be thought of like a burnt CD. Write once, ready many.

The Casper loop filesystem is a fixed-size overlay that allows persistence, or writes, to the LiveUSB. It is normally 4GB or less to comply with file size limitations of FAT32, the normal underlying filesystem for LiveUSB.

I’m not sure how or why it is reporting 8GB, unless the tool you used made an 8GB Casper loop for persistence and you’re not using FAT32…

The drive is formated in FAT32 for sure, it would not boot when formated in exFAT nor NFTS…

Tonight i will try to set up a larger casper file then

1 Like

You can’t exceed 4GB, though, so your problem remains. You might be able to chain-boot from GRUB2 to FAT32 boot volume to exFAT/NTFS/ext4 , but this is getting pretty complicated and where I fell down trying to get a large persistence.

I guess I figured it out!

Used Rufus to flash the .iso, setting up a persistent partition and NFTS as the file system.
Doing that will reflect in a increase of Stored Space reported in the Diagnostic on WebODM. Tried that with 32Gb and 48Gb (didnt want to push my luck cranking it all the way up), an in both cases these values showed up as Storage Space for WebODM to work with

Still haven’t run a dataset as I dont have one at home right now, but will run it tomorrow at work.

2 Likes

Rufus is pretty incredible!

Glad it let’s you go over 4GB on an appropriate filesystem! Other tools still won’t, haha.

Keep in mind that some things can’t up updated on the loop filesystem, like the kernel, so a reflash will likely be in order sooner or later.

1 Like

Turns out it didn’d work… Sort of…
I manage to make it work on legacy mode, but I need to use the usb on some Dell notebooks that does not support legacy boot (Why the hell would they do that?!?)…

I can get to initramfs, but i cant get it to boot on non legacy mode…

Right, if you read the documentation for Rufus, you’ll note that the NTFS module for UEFI isn’t able to be signed for SecureBoot.

You’ll also need to decide if you want it bootable from BIOS or UEFI. They use different disk layouts and boot methods. They are not compatible.