Our Run Models CoLab provides Logistic Regression, Support Vector Machines (SVM), MLP, XGBoost and two Random Forests.
Paths for features and targets are joined based on location IDs (state or county FIPS, etc.) or brain regions for eye blink predictions.
For Features, the UI picks features.dcid if it exists otherwise features.data otherwise the filename from features.path.
For Targets, it picks targets.dcid if it exists otherwise targets.data otherwise the filename from targets.path.
Our main input is currently industry features by county ID (FIPS) for exploring environmental impact targets like bee data and tree canopy.
We are also preping data for International Trade Flow.
run-models-colab.py is the .py export of the Run Models CoLab, kept for reference. The trimmed, runnable version of that notebook is api/run_models.py (see "Running RealityStream Locally" below).
The Run Models colab merges feature and target datasets in Pandas.
Training files can be reviewed by setting save_training=True.
Select one or more models, then paste the resulting yaml in the Run Models CoLab.
Learn about our cuML GPU speed enhancements - and SMOTE balancing of our classes We're using SHAP to explain our model predictions
Bee YAML Updated - Changed bee data target to bees-targets-top-20-percent.csv in parameters yaml. This new "colony density" target uses the top 20% of counties with the highest bee population density (rather than top colony growth between years, as was used in bees-targets.csv).
In the CoLab, a select menu allows you to choose default parameter yaml paths set in parameter-paths.csv.
parameters-simple.yaml - Bee density for 2020 for Maine and New York by top industries (2-digits)
parameters.yaml - Predicts bee density by industry (6-digit NAICS) for New England states 2017 to 2021
parameters-gdc.yaml - Google Data Commons - Predicting states using demographics
parameters-zip.yaml - Needs zip code target. Uses bee population density.
parameters-blinks.yaml - Uses only features dataset (which contains the target column).
TO DO: Top ten counties in each state likely to have declining tree canopy
TO DO: Within Run Models, add python pull from Google Data Commons API for population, education levels, income/poverty levels to use as both features and targets. - IN PROGRESS
TO DO: Use Tensorflow.org for Neural Network predictions with our training data.
api/run_models.py is the colab pipeline as one importable module. It fetches features and targets from a parameters.yaml, trains the requested models twice (plain and SMOTE-balanced), and writes the same report/ folder the colab pushes to modelearth/reports.
pip install -r requirements.txt
python api/run_models.py parameters/parameters.yaml
python api/run_models.py parameters/parameters-blinks.yaml --report-dir report --n-iter 20
python api/run_models.py parameters/parameters.yaml --upload # also push report/ to modelearth/reports
Model keys for the YAML models list: lr, rfc, rbf, svm, mlp, xgboost. rbf is the Random Bits Forest binary, which runs on Linux only (Colab, Docker, Cloud Run); on Windows and macOS it is skipped with a warning.
report/ contains model_performance_report_no_smote.csv, model_performance_report_smote.csv, feature_importance_xgboost.csv, parameters.yaml, model-options.csv, README.md and index.html.
Add an evaluation: block to the parameters YAML to also score with k-fold cross-validation. Omit it and the run is unchanged.
evaluation:
cv: stratified # or: group
folds: 5
stratified uses StratifiedKFold(shuffle=True) over all rows. group uses GroupKFold with each county's state (FIPS // 1000) as the group, so whole states are held out — this exposes geographic leakage (state-held-out scores sit below stratified); with fewer groups than folds it falls back to stratified. Scaling and SMOTE are fit inside each fold (fixed model settings, no nested search, to stay within the Cloud Run cost cap). Results — fold mean ± std for ROC-AUC, PR-AUC, balanced accuracy and macro F1, plus baseline lift — go to model_performance_cv.csv and the "cv" key of the run summary; the existing report files are untouched.
Add split, search or hyperparameters sections to change the test share, the seed, the search size or a model's settings. Leave them out and the run is unchanged. api/model-params.yaml lists every setting with its default and range, and the run stops with a message naming the setting when a value isn't allowed.
split:
test_size: 0.3 # 0.1 to 0.5 (default 0.2)
random_state: 7 # seeds the split, models, SMOTE and search (default 42)
search:
n_iter: 10 # RandomizedSearchCV tries for xgboost and mlp (default 20)
hyperparameters:
rfc: {n_estimators: 200, max_depth: 12}
lr: {penalty: l1, solver: liblinear, C: 0.5}
xgboost: {learning_rate: 0.05} # a fixed value leaves the search; the rest are still searched
Each model in the run summary lists its params_used, including the values the search chose. On the Cloud Run API, changing these settings needs the Team Passphrase.
Settings are read one name at a time: first from the environment, then from the env file named by automation/paths.yaml in the webroot. The whole file is never loaded.
GITHUB_REPORTS_TOKENfor--upload(GitHub token with write access to modelearth/reports)DATACOMMONS_API_KEYfor parameter files that usedcid(Google Data Commons)ENABLE_GPU=trueto use cuML models when RAPIDS is installed (falls back to scikit-learn otherwise)
When a key is missing, the CLI and the Flask /run endpoint explain where to get it.
api/main.py wraps the pipeline as POST /run (body: parameters.yaml text or JSON {"parameters": "parameters-blinks.yaml"}, ?upload=1 to push the report), plus GET /parameters and GET /health. deploy/deploy-cloud-run.sh cpu|gpu deploys it without a Dockerfile: Google Cloud buildpacks read requirements.txt, Procfile and .python-version, and .gcloudignore limits the upload to api/ and parameters/. Setup steps: deploy/. See also AGENTS.md and PLAN.md.
python -m pytest tests