I’ve been working with @calcmogul on improving SysId’s quality-of-fit metrics to better alert the user when data from a routine are undersampled and the calculated gains are unreliable. We continue to see users who cannot interpret the existing diagnostics, and this is not entirely the fault of the users.
A principal component analysis of the data matrix seems like a promising approach for determining which system parameters/control gains can be validly approximated - but interpreting the results of the analysis at the level of concreteness appropriate for the tool has proven challenging (we’re trying to keep this an informative tool and so we’re not happy to just leave it to uninterpretable magic numbers).
We’re making progress, but the we’ve reached the point where the work could use guidance from someone with “proper” expertise in the field (we are both software engineers, not statistical physicists).
If anyone with background in statistical analysis of dynamical systems (or with statistical physics in general) wants to help here, it’d be appreciated - i’m okay with responses here or in DM.