More Campus Resources

Useful Tools and Information

Language

Multilingual content from IBKR

# Towards Better Keras Modeling – Part VI

###### Posted March 25, 2021 at 1:32 pm
The Alpha Scientist

See the previous installment in this series to learn about Univariate Relationships.

### Multivariate Effects

Of course, reality is non-linear and we can’t think about each of these parameters in isolation. Below are a few ways to look at pairs of factors that are logically related.

df_grid = df.reset_index().groupby([“first_neuron”, “hidden_neuron”]).val_loss_improvement.mean().unstack()
ax = sns.heatmap(data=df_grid,cmap = (sns.diverging_palette(10, 220, sep=80, n=7)),annot=True)
ax.set_title(‘Log-Loss Improvement for Varying Shapes’)

# Unsurprisingly, bigger on both layer types is better.

df_grid = df.reset_index().groupby([“hidden_layers”, “hidden_neuron”]).val_loss_improvement.mean().unstack()
ax = sns.heatmap(data=df_grid,cmap = (sns.diverging_palette(10, 220, sep=80, n=7)),annot=True)
ax.set_title(‘Log-Loss Improvement for Varying Shapes’)

df_grid = df.reset_index().groupby([“dropout”, “hidden_neuron”]).val_loss_improvement.mean().unstack()
ax = sns.heatmap(data=df_grid,cmap = (sns.diverging_palette(10, 220, sep=80, n=7)),annot=True)
ax.set_title(‘Log-Loss Improvement vs Dropout/Size’)

# Surprisingly, dropout didn’t help even at larger geometries

metric = ‘hidden_layers’
ax = sns.boxplot(x=metric, y=”val_loss”, data=df.reset_index(),color=’salmon’)
ax.set_title(f’Validation Loss as function of {metric}’)

# Very interestingly, no hidden layers is MUCH better. Possibly a consequence of
# some deep models with too few neurons to properly represent data

Stay tuned for the next installment in this series to learn more about Relative Importance of Features.