I’ve written a lot of redundant code to control an experimental apparatus with python.
The Cartesian product almost solves everything, but there’s still one hickup I need help with.

Here’s the story:

Usually, I need do a grid search over multiple parameters.
Occasionally I’ll need something slightly more general, which I’ll cover at the end.

I can do this with deeply nested for loops:

for param1 in np.linspace(...):
  for param2 in np.linspace(...):
    for param3 in np.logspace(...):
       ... # more for loops if more params swept

          obj = defaultvalues() # some class elsewhere defined
          obj.voltage2 = param1
          obj.current3 = param2
          obj.current98 = param3
          ... # more lines if more params swept
          
          saveResult(          # placeholder for some code that saves the data
             apparatus(obj)    # runs a real life experiment, and returns measurements.
             )

The issue with this is that it’s:

1. a lot of redundant code
2. slow to reconfigure
3. And the code almost needs to be saved along with the data to know what the data is.

Cartesian product solution

obj = defaultvalues()
obj.voltage2 = np.linspace(...)
obj.current3 = np.linspace(...)
obj.current98 = np.logspace(...)

for testObj in cartesianProduct(obj): #sloppy with notation
   saveResult(          # placeholder for some code that saves the data
      apparatus(obj)    # runs a real life experiment, and returns measurements.
      )

(there’s probably a better way to do this by defining a factory method in defaultvalues)
This is a lot better, and I could replace the first few lines with a gui (which just prompts the user to provide a list for each parameters they want to sweep.).

Additional needs

order of parameters swept

the order that I sweep parameters maters in real life, so I’d actually need to specify an order.
That’s fine though, I’d just need a bit of extra code to change what order to plug those lists into cartesianProduct

initialization of parameters (the actual hiccup)

sometimes I need to apply a set of values to the aparatus, (without needing to measure) to prepare the aparatus for some other measurments.
Editing the for loops solution, it would look like:

for param1 in np.linspace(...):
  for param2 in np.linspace(...):
    for initializer1 in [-1.0,1.0]
       obj = default(obj)
       ...                       # set the other param values from for loop
       obj.magneticField=initializer1     # set the magnetic field away from default
       obj.current98= 1300   #temporarily set the value to initialize the system
       apparatus(obj)            # don't need to save the measurement results here
     
       for param3 in np.logspace(...):
           ... # more for loops if more params swept

              obj = defaultvalues()
              obj.voltage2 = param1
              obj.current3 = param2
              obj.current98 = param3
              ... # more lines if more params swept
          
              saveResult(         
                 apparatus(obj)   
                 )

Solution to additional needs

I’m imagining that the ideal solution would be that the user type something like the following into a file, or interacts with a gui to input the following, or writes a small bit of python that looks like the following, and then can run the experiment.

voltage2 = np.linspace(...)
current3 = np.linspace(...)
MagneticField: [-1.0, 1.0]: initialize
Current98: 1300: initialize
current98: np.linspace(...)

This should do the same thing as the above for loop based code.

Note that this information would be the same information as the metadata for a python xarray
that contains the result of the grid search sweep.
Except for the initialization part.

The more general case:

You could write this out algebraically as the tensor product of vectors.
v1 … vi would represent the different values of the voltage that you want.
i1, i2 would represent different values of the current, and so on.

(v1 + v2 + v3) * ( i1 j1 + i2 j2)  * (initializeB1 +initializeB2) (...)

And the resulting terms would each look like

v1*i1*j1*initializeB2 ...

Which would each represent a combination of parameters that could be run on the apparatus.

This is slightly more general because it allows for coordinated entries like (i1 j1 + i2 j2).

The Question

This seems like any software controlled experiment would have to manage this exact same problem.
This could be done most generally with a custom code file for each experiment sweep configuration, one based on nested for loops, but in my experience that has been prone to user error.

I’ve made some suggestions here about possible solutions to the problem.
What I’d like to know is:

1. best practices
2. if there is already a package/packages that do this (preferably ones that also manage the data storage into something user friendly, like an x-array)
3. If there are other solutions to this problem
4. If there is an easy way to implement my suggested solution to the problem