GPU usage development

Developers Chat
#1

Ok, just about ready to start compiling. I plan to proceed in an orderly fashion, slow but starting from a working base. So,

phase 0. become familiar with the algorithms inherent in the ODM code, especially the ones that take hours to complete. :slight_smile:

phase 1. Would be nice to get the compile instructions for each program, sans Docker. I would like to run the first tests natively for several reasons, including a) unfettered benchmark comparisons to the ones already published, and b) never developed in Docker so donā€™t know itā€™s requirements separate from bare metal.

phase 2. Then, once the programs are compiled and verified as working correctly, switch over to Qt Creator so I can interactively test sections of code to find the actual bottlenecks. Which functions, are we I/O bound, compute bound, or what.

phase 3. Start incorporating the Cuda instructions. 3a. verify each function with the code changes to ensure they are still operating correctly and give the right information.

phase 4. Look at the package overall and see if there is any gain to passing/leaving work files for the next program to pick up and use. Not likely, but itā€™s in the path.

phase 5. Release pre-alpha for you guys to test. Pre-alpha will require an nVidia graphics GPU and Linux. Itā€™s what I have so just need to proof it before striking out to supporting other GPUā€™s, Windows, and etc.

phase 6. Expand utility to use by other sytems and GPUā€™s. (1st alpha)

phase 7. If anyone wants to, include the ability to spread their workload

phase 8. Determine if it is possible and makes sense to reduce the need for mega-memory in your computer to get good runs. User will need large, fast disks. The larger and faster, the better. (Ainā€™t it always so?) :slight_smile:

phase 9. Trying to incorporate feed back and bug reports as we go along, and hopefully release 1st beta.

Sound like a plan?

No matter what, I need the information to 1) not use docker and 2) get (non-docker) compile parameters for each program, OpenSFM, ODM, etc. Please.

David

2 Likes
#2

Rats. Just discovered some of the existing codebase is in C, some in Python. Not a show stopper but will take a bit longer. I thought it was all Python.

1 Like
#3

Ok, ran ā€˜start-dev-env.shā€™ first. Got some errors. Then ran ā€˜configure.shā€™ and got more errors. (Ran in that order.) Took a bit of effort with both to get the arguments lined up. :slight_smile: Maybe I didnā€™t get them right but most of each ran.
Iā€™ve saved the screen output from ā€˜start-devā€™ and of course, have the config log. Would it be asking too much to get one of you experienced guys to look at them and tell me how to run them without errors? They are a bit hefty each. Not sure if yā€™all would want the files uploaded in here.

Thanks,
David

2 Likes
#4

Installation should be pretty straighforward on Ubuntu 16.04:

Thatā€™s the best place to start. Work is needed to fix dependencies on newer versions of ubuntu.

1 Like
#5

Iā€™m trying :sob:

image
image704Ɨ728 33.8 KB

But yeah, if you need testing, my home machine(s) are i7-6700k with GTX 1050TI OCs running Windows 10 Pro x64, and I have a bootable LiveODM 16.04 image as well.

1 Like
#6

Much appreciated too!

1 Like
#7

All 3 desktops (4 if I can fix one) run Debian. 3 are on stretch, the newer one on Buster. I despise ubuntu even though nVidia -forces- us to use it on the Jetsonā€™s. I hope you guys can give me clues anyway 'cause Iā€™d hate to try and go it alone.

David

#8

Canā€™t quite follow, in cmake files, VTKconfig.cmake and vtk-config.cmake or their (conflicting?) purpose. True, Iā€™ve never used VTK before; we usually had to make do with our own GL constructions from scratch but I left those behind in the 80ā€™s.

1 Like
#9

Smather, Could you, or anyone, please supply a really simple list of the order in which your shell install scripts need to be run while trying to install from scratch? Studying the scripts has revealed the individual task arguments available per task, but in what order do we run the (very long) scripts to install properly?

1 Like
#10

Btw, I caved and bought an 8 TB hard drive. Put 7 TB into itā€™s own partition, installed Ubuntu (18 was not available so had to go with 20,) and installed just about everything so far. That said, still unclear how to install ODM, or even which VERSION of ODM to install for GPU development: nodeODM, WEBODM, or what? Installed Docker but still donā€™t want to use it.

TAP TAP TAP. Is this thing on?

1 Like
#11

So, I believe youā€™ll need ODM and WebODM installed/compiled locally at the bare minimum. Iā€™ve yet to be able to do this successfully on anything other than 16.04 LTS

#12

As far as I can tell, NO sources, .py or .c, downloaded from the github zip file. Is this normal?

#13

The installation script itself downloads the dependencies.

1 Like
#14

Can you tell me in what order the scripts are to be run? Or does it matter?

#15

And is there anything in any of the code that absolutely -requires- Docker? API calls or anything?

#16

No docker requirement in the toolchain.

1 Like
#17

Thereā€™s only one installation script.

1 Like
#18
  1. dev-env.sh <ā€” is this run before or after install?
  2. dev-worker.sh
  3. wait-for-it.sh
  4. wait-for-postgres.sh
  5. webodm.sh
  6. start.sh <----- is the installation command in here (I forget)
  7. worker.sh
    Only one of these 7 scripts is obvious. well, maybe 2, counting postgres. :slight_smile:

David

1 Like
#19

Gotcha. I assumed that you would start with just OpenDroneMap:

In which case you would look at configure.sh

#20

Raw build, no docker, native and needing ALL relevant source code.
Steve, the ā€œconfigure.shā€ is an extremely complicated and multi-level build script! Slowly tracking down the failures and eliminating the errors one by one. This one I need a bit of info on: the ppa:dennis-somebody Cmake repository ā€“ is it still needed? With all the improvements in ā€œstockā€ cmake 3.7 and 3.8, is the dennis one necessary, or is something else going on the I am unaware of yet? That repo link is no longer available. I finally found what I think is itā€™s replacement, butā€¦

Thanks,
David

1 Like