Grasshopper

algorithmic modeling for Rhino

parallel computation - run iterations on multiple computers?

Hi all,

 

I want to take advantage of all the unused desktops that my company has sitting around at night. I'm searching for a way to create something like a render farm or server computing through GH.

 

When using GH as a translator to run multiple iterations through 3rd party programs like Radiance or E+ (see honeybee modules), the calc doesn't live in GH/rhino. GH simply calls these programs. Unfortunately, every iteration is called in series. Once one ends, the next begins. I'd like to send these iterations to multiple computers on a server so that I can run in parallel and reduce my wall-clock time.

 

I'd like to generate all 500 batch files from one machine, then somehow call those routines on other computers on my network, say 100 runs on 5 machines. Or 5 runs on 100 machines (even better!). I hope this is possible because we are not splitting any single computation between multiple machines. We only want to call unique runs across a network. 

 

Has anyone explored this? If we figure this out, companies or students could take advantage of all the unused desktops in offices or computer labs to calculate large sets overnight. Which would be amazing.

 

Thanks for your help.

-Leland

Views: 2508

Replies to This Discussion

Hmm ... assuming (by the question) that you are familiar with code matters get this very entry level simple test case and we can further "elaborate" ... or not.

Note: // computing is a giant pile of worms. Actually it's a colossal pile of worms (without the proper scheduler. BTW: are your iterations asynchronous?).

Attachments:

Peter,

 

Thanks for responding. I am not an expert with code matters but I know what I need. My iterations are asynchronous, which I hope will simplify things. Once the iterations have been properly parceled using your code or something similar, how can the resulting batch files be sent and executed by a set of clustered computers?

Very interesting stuff! How sophisticated do you want to be? If you don't mind setting each machine up manually (e.g. your five machines and 100 runs example) you could use the RunItAll component with the sliders on each machine's GH file set up to run their own unique slider combinations. Then, once all machines are finished, you can collate results files manually. This has worked well for me in the past.

A more automated workflow (necessary for 100 machines and 5 runs!) would be much more challenging. For one, I'm not sure if there's even a way to open Grasshopper and a specified file from the command line (correct me if I'm wrong!). 

Presumably Rhino licensing is not an issue if you were able to get this working?

There is in fact a way to launch rhino + GH with a particular file. This is the approach we used on pollination, back at the 2014 AEC hackathon in NY. All that code is available (though it's not especially well documented, so use at your own risk). https://github.com/HoneyPatch/pollination

The general approach with launching Rhino + GH around a file is to use a .bat file with the following command: 

@ECHO OFF
cd **PATH TO DIRECTORY CONTAINING GH FILE**

"**PATH TO RHINO.EXE**" /nosplash /runscript="-grasshopper editor load document open **GRASSHOPPER FILE NAME** _enter" "**PATH TO ASSOCIATED RHINO FILE**"

on pollination, if memory serves, we were actually launching all identical instances of Rhino w/ grasshopper opening the SAME file - but the GH file looked at a .json file containing all the parameters, and we auto-generated different json files for each parameter set we wanted to run. 

Hi Andrew,

Thank you for your amazing and inspiring Pollination project! That was a indeed a milestone in gh cloud computing!

I noticed that you have system_0, 1, 2 in your pollinate/instances folder. Does it mean that you're using three instances during the Hackathon?

So did you manually install Rhino and grasshopper on all the instances?

I found that the following link might be helpful to install all Rhino and GH automatically:

http://wiki.mcneel.com/rhino/installingrhino/5.0

and

https://gist.github.com/padgo/984f560e7ff46f2f5a15

But I haven't tested yet.

I don't know if we can bypass the licensing process and just do it in trail version.

(Especially when you are using AWS, you won't buy license for hundres of computer really)

What do you think of it?

RSS

About

Translate

Search

Photos

  • Add Photos
  • View All

Videos

  • Add Videos
  • View All

© 2024   Created by Scott Davidson.   Powered by

Badges  |  Report an Issue  |  Terms of Service