Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
291 commits
Select commit Hold shift + click to select a range
2644296
names
jpn-- Jul 6, 2021
9788123
to tour mode
jpn-- Jul 6, 2021
59f562a
tinker names
jpn-- Jul 6, 2021
f1bb837
just pretty
jpn-- Jul 6, 2021
b0c7dba
generic cleaning
jpn-- Jul 8, 2021
f9ca9c2
Merge branch 'condaforge-dev' into performance1
jpn-- Jul 17, 2021
5df7b42
define mtc1 as 20K households
jpn-- Jul 17, 2021
43723e1
ready for win
jpn-- Jul 17, 2021
c91f17c
pb branches
jpn-- Jul 18, 2021
50c24ba
Merge branch 'faster-maz-maz' into performance2
jpn-- Jul 18, 2021
d059d3b
Merge branch 'speeding-up-chunking' of https://github.com/ActivitySim…
jpn-- Jul 18, 2021
20ec9f1
more aggressive chunkless
jpn-- Jul 19, 2021
e4e5657
asv commit logging
jpn-- Jul 19, 2021
27bab07
Merge branch 'master' into performance2
jpn-- Sep 7, 2021
d90de76
Merge branch 'checksum-downloads' into performance2
jpn-- Sep 7, 2021
675f588
asv sandag1
jpn-- Sep 8, 2021
761319f
setup_cache as func
jpn-- Sep 8, 2021
1a7ba61
fix sandag1 benchmarks
jpn-- Sep 8, 2021
4e7d041
allow other configs dirs in prerun
jpn-- Sep 8, 2021
4bacdce
data and output definable
jpn-- Sep 8, 2021
e677f09
checksums were correct
jpn-- Sep 8, 2021
4e19b84
dirs in component setup
jpn-- Sep 8, 2021
e7eaaed
only use 20K hh for sandag tests
jpn-- Sep 8, 2021
6484ebd
documentation updates
jpn-- Sep 8, 2021
1c13c61
sandag2
jpn-- Sep 8, 2021
91cc417
submitting benchmarks
jpn-- Sep 8, 2021
ff0345c
fix sandag2 benchmarks
jpn-- Sep 8, 2021
dae575b
note about escapes
jpn-- Sep 8, 2021
b7fa769
pycodestyle
jpn-- Sep 8, 2021
579917e
sandag3
jpn-- Sep 9, 2021
1db6e1c
extra cli run args
jpn-- Sep 13, 2021
cbe00f8
extra logging
jpn-- Sep 13, 2021
6c52cd4
performanceTest
jpn-- Sep 13, 2021
db11669
elapsed time logging
jpn-- Sep 13, 2021
064942e
elapsed time logging
jpn-- Sep 13, 2021
52887e6
use cached accessibility
jpn-- Sep 13, 2021
b7beffa
settings remap
jpn-- Sep 14, 2021
93db698
Merge commit '6ba7e8ff386c95e3bf1b05864cc42461f1e953aa' into performa…
jpn-- Sep 14, 2021
47ed335
print line
jpn-- Sep 14, 2021
4359db0
profiling
jpn-- Sep 15, 2021
4708658
fix branch names
jpn-- Sep 15, 2021
d289734
use smaller sandag 2-zone
jpn-- Sep 15, 2021
37498ed
mp complete
jpn-- Sep 17, 2021
3d6a5b5
profile inspector
jpn-- Sep 17, 2021
f058751
sandag2 mp complete
jpn-- Sep 17, 2021
9f91840
Merge commit 'fd077d644fdb41200031f2860cfc0d60e2b90574' into performa…
jpn-- Sep 17, 2021
3b297aa
pipeline hash for benchmarking
jpn-- Sep 17, 2021
3a1331c
absolute import
jpn-- Sep 17, 2021
b515c02
manifest points to a place
jpn-- Sep 17, 2021
c687cc0
allow remote manifest files to be gzipped, then expanded after download.
jpn-- Sep 17, 2021
4e63ded
harmonize sandag3
jpn-- Sep 17, 2021
c6b336a
sandag1
jpn-- Sep 18, 2021
878e100
commit token
jpn-- Sep 19, 2021
55b9980
extra logging, stable option for prerun
jpn-- Sep 20, 2021
9cb9860
remove pipeline hash
jpn-- Sep 20, 2021
e41084d
sandag1e
jpn-- Sep 20, 2021
eb08c05
mostly templated sandag
jpn-- Sep 20, 2021
fe66db5
cleanup
jpn-- Sep 20, 2021
f9ec119
sandag3f
jpn-- Sep 20, 2021
67a18d5
use full population in data for MP complete test
jpn-- Sep 20, 2021
cf722a8
benchmark latest
jpn-- Sep 20, 2021
4949d90
benchmarking configs
jpn-- Sep 20, 2021
f352207
allow missing models
jpn-- Sep 20, 2021
0590bbc
sandag2f
jpn-- Sep 20, 2021
d937ecc
batch
jpn-- Sep 21, 2021
04129e0
update mp_complete name
jpn-- Sep 22, 2021
3637823
rename to fuller names
jpn-- Sep 23, 2021
4efd4ab
skip a few
jpn-- Sep 23, 2021
350aeb4
farm out setup cache
jpn-- Sep 24, 2021
403ec3b
template in componentwise
jpn-- Sep 24, 2021
5a536ae
normal logger
jpn-- Sep 24, 2021
932ee91
cwd error
jpn-- Sep 24, 2021
2169c7a
fix dirs
jpn-- Sep 24, 2021
a00741c
template mp complete
jpn-- Sep 24, 2021
7216304
template complete does not work
jpn-- Sep 24, 2021
bd41fc8
one ping only
jpn-- Sep 24, 2021
720d03e
fix sandag 2e
jpn-- Sep 24, 2021
2dfd76a
blacken
jpn-- Sep 24, 2021
9f1e0e4
cwise template blackened
jpn-- Sep 24, 2021
9d92158
remove cwise template
jpn-- Sep 24, 2021
7c55807
fix mtc1full benchmark
jpn-- Sep 27, 2021
8dc1924
clean old benchmark files
jpn-- Sep 27, 2021
b0e3447
hook to fix paths in benchmark output
jpn-- Sep 27, 2021
241ddb6
some docs
jpn-- Sep 27, 2021
0384d2c
documentation
jpn-- Sep 28, 2021
e936ec6
writing new benchmarks doc
jpn-- Sep 28, 2021
dac52e6
explicit benchmark versioning
jpn-- Sep 28, 2021
9be2832
use std name
jpn-- Sep 28, 2021
cbc6575
consistent single-thread benchmark size
jpn-- Sep 28, 2021
d4fe36a
tracking for mp benchmarks
jpn-- Sep 28, 2021
d419e06
fix log file path
jpn-- Sep 28, 2021
418c479
fix track read
jpn-- Sep 28, 2021
3b4b80f
clean up
jpn-- Sep 28, 2021
18128f2
minimally functional multithread benchmark
jpn-- Sep 29, 2021
7c3053d
todo note for MP bench
jpn-- Sep 29, 2021
6d63e7a
note about MP benchmarks
jpn-- Sep 29, 2021
bf64828
pycodestyle
jpn-- Sep 29, 2021
5ec18ed
remove config editing
jpn-- Sep 29, 2021
8bddc97
strip ruamel, not needed
jpn-- Sep 29, 2021
a01fcd1
fix to use dynamic configs dir
jpn-- Sep 29, 2021
b0e8495
Merge branch 'develop' of https://github.com/ActivitySim/activitysim …
jpn-- Nov 8, 2021
d1e73b6
use different branch
jpn-- Nov 8, 2021
7ce54aa
Merge commit 'd3ee01edaf17b093c14e559f45a029eb47c2ee5d' into performa…
jpn-- Nov 19, 2021
e3e4e33
benchmarking docs
jpn-- Nov 19, 2021
91f3f3c
point to final asim-benchmarks
jpn-- Nov 20, 2021
66210f9
Merge commit '3df695bd2bf921aa46d296a09889b68087b8c911' into performa…
jpn-- Nov 21, 2021
24c7db9
doc formatting cleanup
jpn-- Nov 22, 2021
dbee4cd
asv continuous
jpn-- Nov 22, 2021
5142bd7
include benchmarking in docs table of contents
jpn-- Nov 30, 2021
97ca9ac
First summarize commit.
Nov 11, 2021
8323213
Quick concept test for summarize
Nov 11, 2021
44be451
generalized output path
chesterharvey Nov 12, 2021
981803d
Added functionality for summarize to access pipeline if available or …
chesterharvey Nov 16, 2021
ca3b205
Initialize test via pipeline
Nov 18, 2021
a24e754
update testing location
Nov 18, 2021
85abe6a
organize mock pipeline.
Nov 18, 2021
f32525d
update for network_los as an injectable.
Nov 30, 2021
d1c8f42
added logic to add output directory if not already available before i…
chesterharvey Nov 18, 2021
e5e8e14
Added logging for summaries
chesterharvey Nov 18, 2021
828d973
generalized summarize model so that all tables are available as local…
chesterharvey Nov 30, 2021
7e187f4
added annotate_trips to summary function--needs work
chesterharvey Nov 30, 2021
6e9c804
built summaries for demo dashboard
chesterharvey Nov 30, 2021
de289e9
updates for network_los
Nov 30, 2021
2fbdfed
added skim summary columns to trips in summary function
chesterharvey Dec 1, 2021
9910dfd
added simwrapper yamls and summarize step on model run
chesterharvey Dec 2, 2021
0167503
added simwrapper templates to example_mtc output
chesterharvey Dec 2, 2021
d286b50
updates to summary expressions and simwrapper yamls
chesterharvey Dec 2, 2021
3d291f1
Add slicers and aggregators. Intermediate check-in. Need to re-attach…
Dec 7, 2021
38ae31b
Reimplement initial skims based calculations. more work needed.
Dec 7, 2021
b8a7128
Add preprocessor to summarize.
Dec 8, 2021
3e7c1d2
changed key name for bin breaks
chesterharvey Dec 8, 2021
09bae50
Add preprocessor
Dec 8, 2021
eacdfa9
summarize preprocessor expressions
chesterharvey Dec 10, 2021
a6e461f
Update summarize_preprocessor.csv
chesterharvey Dec 10, 2021
2d564d0
Update summarize_preprocessor.csv
chesterharvey Dec 10, 2021
de4e62a
updated summarize preprocessor expressions
chesterharvey Dec 10, 2021
0f811ab
summarize preprocessor expressions
chesterharvey Dec 10, 2021
0a49cc2
summary expression updates
chesterharvey Dec 10, 2021
6c84299
added summarize preprocessors
chesterharvey Dec 11, 2021
20268da
updated summarize preprocessor expressions
chesterharvey Dec 11, 2021
2e5157d
updated summarize configs in example_mtc model
chesterharvey Dec 14, 2021
3e877bd
Added binning functions for preprocessing and summary expressions
chesterharvey Dec 15, 2021
15c3a6a
added example summaries incorporating slicers
chesterharvey Dec 16, 2021
243c9ed
remove old csv file....
Dec 22, 2021
a51a201
remove extraneous print statement
Dec 22, 2021
dfbae13
black + isort
Dec 22, 2021
92a20df
Type hints
Dec 22, 2021
aa4c528
Add type hints
Dec 22, 2021
211e1d8
Change debug level
Dec 22, 2021
3cd363d
test data
Dec 22, 2021
04f3ed2
Initial docstrings
Dec 22, 2021
8e6bcf0
code cleanup
Dec 22, 2021
186ae09
Re-orient tests from root directory
Jan 4, 2022
a5be56f
Reorient tests
Jan 4, 2022
ce85d26
Update Sphinx docs
Jan 4, 2022
6b6055e
fix pycodestyle
Jan 4, 2022
80310a8
Fix binning overflow.
Jan 4, 2022
5356a19
Constrain Numpy version #533
Jan 4, 2022
7c704ed
Remove write_summaries from marin example
Jan 4, 2022
0ec3c47
Docstrings for summarize.py methods
chesterharvey Jan 5, 2022
89013ce
aggregation tests added for summarize model
chesterharvey Jan 5, 2022
0c025b1
pycodestyle cleanup
chesterharvey Jan 5, 2022
a2023b4
fixed typo in example_MTC summarize.yaml
chesterharvey Jan 6, 2022
9eb136d
Update LICENSE.txt
Jan 10, 2022
74569ce
Merge pull request #534 from ActivitySim/license_update
AMPO-Staff Jan 14, 2022
d6afb46
add doc for benchmark batch
jpn-- Jan 31, 2022
6efb433
dynamically get current branch for asv
jpn-- Jan 31, 2022
e65710c
some high level information on what Benchmarking does
jpn-- Jan 31, 2022
5683051
need to sync to publish
jpn-- Jan 31, 2022
2f580b8
branch arg
jpn-- Jan 31, 2022
d5eb981
add docs
jpn-- Feb 1, 2022
4a93836
Merge commit '74569ceae1f0f878de75ea328a0a0115abddeb97' into performa…
jpn-- Feb 1, 2022
fa6ecc6
constrain numpy
jpn-- Feb 1, 2022
b22e978
initialized testing structure for auto ownership mode
chesterharvey Feb 7, 2022
68a65a2
made all standard pipeline tables available for summary expressions
chesterharvey Feb 10, 2022
d9277a5
syntax big fix
chesterharvey Feb 10, 2022
eb36fa5
updated functionality for exporting available pipeline tables for exp…
chesterharvey Feb 10, 2022
60ca801
configured export of pipeline tables as a yaml flag
chesterharvey Feb 10, 2022
f551dd1
setup testing framework for auto_ownership model
chesterharvey Feb 14, 2022
d54948e
pass through kwargs for estimation on location models
jpn-- Feb 22, 2022
aa69037
numpy max 1.21
jpn-- Feb 22, 2022
91942ea
extra larch dependencies for asim estimation
jpn-- Feb 23, 2022
a66e265
fixed code style errors
chesterharvey Feb 23, 2022
60fd31c
allow annotate_preprocessors to annotate a table without a skim wrapper
chesterharvey Feb 23, 2022
f812657
defined separate preprocessors for different pipeline tables
chesterharvey Feb 23, 2022
0458d45
enabled preprocessing for multiple tables
chesterharvey Feb 23, 2022
83a9522
added ability to create temporary variables in summary expressions file
chesterharvey Feb 24, 2022
0deec59
added the simwrapper python package as a dependency
chesterharvey Feb 24, 2022
78dd56a
fixed python style
chesterharvey Feb 24, 2022
831d58c
updated summary expressions
chesterharvey Feb 24, 2022
2f2dbe3
add Dockerfile
i-am-sijia Feb 24, 2022
e53c99c
added summarize model to example_mtc
chesterharvey Feb 24, 2022
bac92a8
updated visualization docs
chesterharvey Feb 24, 2022
c711bdf
updated docs
chesterharvey Feb 24, 2022
cc9823a
reverting to earlier commit to address travis testing failures
chesterharvey Feb 24, 2022
1dc1aa0
updates to make all pipeline tables available as locals and allow tem…
chesterharvey Feb 25, 2022
38be435
updates to viz documentation
chesterharvey Feb 25, 2022
0b68119
pycodestyle fix
chesterharvey Feb 25, 2022
85c9f04
viz documentation updates
chesterharvey Feb 25, 2022
8198d5b
remove unnecessary test pipeline tables
chesterharvey Feb 25, 2022
9e69428
Add expressions for tours and trips counts
chesterharvey Feb 25, 2022
a30f160
Allow yamls to maintained in example outputs
chesterharvey Feb 25, 2022
aec98f1
Update dashboard-1-summary.yaml
chesterharvey Feb 25, 2022
b17943b
Add summarize model to example_mtc settings
chesterharvey Feb 25, 2022
bb9ad1f
Add simwrapper as a dependency
chesterharvey Feb 25, 2022
9bb224c
added summarize config files to mtc example
chesterharvey Feb 25, 2022
a898568
Updates to docs
chesterharvey Feb 28, 2022
51728e2
rename 'slicers' as 'bins'
chesterharvey Feb 28, 2022
279f060
Allow export of pipeline tables
chesterharvey Mar 1, 2022
139aa1b
Update SLICERS to BIN
chesterharvey Mar 1, 2022
94e655b
Merge pull request #540 from jpn--/numpy-pin
jpn-- Mar 1, 2022
d84e4a1
pass through kwargs for estimation on location models
jpn-- Feb 22, 2022
d6b6d68
Merge branch 'args-on-location-est' of https://github.com/jpn--/activ…
jpn-- Mar 1, 2022
227908f
pycodestyle updates
chesterharvey Mar 1, 2022
79d88f5
removed unnecessary geojson from sample data
chesterharvey Mar 2, 2022
1b1a4c6
Added simwrapper to environment yamls temporarily with pip
chesterharvey Mar 2, 2022
3974e39
Enabled pipeline table export for expression development
chesterharvey Mar 2, 2022
15235e8
add docker-compose
i-am-sijia Mar 3, 2022
d96c740
Merge branch 'mtc_tm2' into ft_vis_1
i-am-sijia Mar 3, 2022
b9ecce4
Revert "Merge branch 'mtc_tm2' into ft_vis_1"
i-am-sijia Mar 3, 2022
0791b4a
Update summarize.csv
chesterharvey Mar 3, 2022
e6f7077
Merge branch 'develop' into ft_vis_1
chesterharvey Mar 3, 2022
afbfe4b
update badge link to travis-ci.com
chesterharvey Mar 3, 2022
d76a94f
fixed bug in travis badge url
chesterharvey Mar 3, 2022
cd8fbad
simwrapper is on conda-forge now, so remove pip install rules & docs
billyc Mar 6, 2022
812186f
Merge pull request #3 from billyc/ft_vis_1
i-am-sijia Mar 7, 2022
8110573
add back pip install -e
i-am-sijia Mar 7, 2022
5495553
Update gettingstarted.rst
jpn-- Mar 8, 2022
c5f437e
Merge pull request #539 from jpn--/args-on-location-est
jpn-- Mar 15, 2022
8ef72ce
require xarray and sharrow for larch testing
jpn-- Mar 15, 2022
1a93cd3
add docstring for template_setup_cache
jpn-- Mar 31, 2022
ef05447
Merge pull request #532 from wsp-sag/ft_vis_1
jpn-- Apr 12, 2022
b3d1d61
Merge branch 'develop' into performance1
jpn-- Apr 12, 2022
4779573
Update test_larch_estimation.py
jpn-- Apr 21, 2022
ec98fc3
pycodestyle
jpn-- Apr 21, 2022
91930b5
Merge pull request #456 from jpn--/performance1
jpn-- Apr 25, 2022
367cbac
🚀 [feat] vehicle type model (#486)
mxndrwgrdnr Jun 6, 2022
f79341d
Differentiate examples between quality and validity of example models…
jpn-- Aug 3, 2022
3b7e551
black and isort (#578)
jpn-- Aug 4, 2022
468270b
stop emitting output dir
jpn-- Aug 6, 2022
74b1465
trace files in nested directories
jpn-- Aug 7, 2022
3b12814
swap files for xborder model
jpn-- Aug 7, 2022
e92ac72
repair ARC MP
jpn-- Aug 8, 2022
93650ab
print checksum even when not used
jpn-- Aug 9, 2022
ee6c9e6
add hashes for sandag_xborder_full
jpn-- Aug 9, 2022
e097c70
fix dtype in university hack
jpn-- Aug 9, 2022
7481f73
fix persons to match tours
jpn-- Aug 9, 2022
1176399
repair ARC
jpn-- Aug 10, 2022
e4910e5
Bump version: 1.0.4 → 1.1.0
jpn-- Aug 11, 2022
0412a0b
Merge pull request #580 from camsys/small-fixes
jpn-- Aug 11, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Added logging for summaries
  • Loading branch information
chesterharvey authored and Clint Daniels committed Dec 22, 2021
commit e5e8e14c8513527c1777151f6c89bc0eadedcfb5
4 changes: 4 additions & 0 deletions activitysim/abm/models/summarize.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ def summarize(network_los):
output_location = segment['output'] if 'output' in segment else 'summaries'
os.makedirs(config.output_file_path(output_location), exist_ok=True)

logger.info(f'Running summaries for {table}')

# Go get specified tables from the pipeline unless they are supplied as a parameter
locals_d = {
table: pipeline.get_table(table)
Expand All @@ -38,6 +40,8 @@ def summarize(network_los):
out_file = row['Output']
expr = row['Expression']

logger.info(f'Summary: {expr} -> {out_file}.csv')

resultset = eval(expr, globals(), locals_d)
resultset.to_csv(config.output_file_path(os.path.join(output_location, f'{out_file}.csv')))

4 changes: 3 additions & 1 deletion activitysim/abm/test/test_misc/test_summarize.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
### import models is necessary to initalize the model steps with orca
from activitysim.abm import models
from activitysim.core import pipeline
import logging


# Used by conftest.py initialize_pipeline method
Expand All @@ -23,5 +24,6 @@ def initialize_network_los():
return True


def test_summarize(initialize_pipeline):
def test_summarize(initialize_pipeline, caplog):
caplog.set_level(logging.INFO)
pipeline.run(models=['summarize'])