Python’s Itertools offers a great solution when you want to do a grid-search for optimal hyperparameter values, -or in general generate sets of experiments-.
In the code fragment below we generate experiment settings (key-value pairs stored in dictionaries) for all combinations of batch sizes and learning rates.
import itertools # General settings base_settings = {'epochs': 10} # Grid search grid = { 'batch_size': [32, 64, 128], 'learning_rate': [1E-4, 1E-3, 1E-2] } # Loop over al grid search combinations for values in itertools.product(*grid.values()): point = dict(zip(grid.keys(), values)) # merge the general settings settings = {**base_settings, **point} print(settings)
output:
{'epochs': 10, 'batch_size': 32, 'learning_rate': 0.0001} {'epochs': 10, 'batch_size': 32, 'learning_rate': 0.001} {'epochs': 10, 'batch_size': 32, 'learning_rate': 0.01} {'epochs': 10, 'batch_size': 64, 'learning_rate': 0.0001} {'epochs': 10, 'batch_size': 64, 'learning_rate': 0.001} {'epochs': 10, 'batch_size': 64, 'learning_rate': 0.01} {'epochs': 10, 'batch_size': 128, 'learning_rate': 0.0001} {'epochs': 10, 'batch_size': 128, 'learning_rate': 0.001} {'epochs': 10, 'batch_size': 128, 'learning_rate': 0.01}