Downloading data from S3 Bucket

This dataset is available as part of the AWS Open Data Program, therefore egress fees are not charged to either the host or the person downloading the data.
Note, this dataset contains the CMAQ 2019 Modeling Platform with inputs and scripts that are configured for CRACMM2 in CMAQv5.5 but these same inputs can be used for CRACMM version 1 or 2 and CMAQ v5.4 with minor modifications.using the CRACMMV1 emissions, epic files, and icbc's.
Link to the bucket described here:
cmaq-2019-modeling-platform
Link to dataverse entry for the data contained within this bucket: CMAQ Model Version 5.4-5.5 CRACMM Input Data -- 1/1/2019 - 12/31/2019 12km CONUS Dataverse

The 2019 met inputs are available under the EQUATES S3 bucket:
EQUATES 2019 MCIP Input

Instructions for downloading the aws command line.

Instructions to Download AWS Command Line
Once you have it installed, you can use it without credentials.
Example commands:
To query how much data is in a bucket:

aws s3 --no-sign-request ls --summarize --region=us-east-1 --human-readable --recursive s3://cmaq-2019-modeling-platform

Output:

2025-01-22 17:17:42  813.8 KiB cmaq-2019-modeling-platform/data/output_CCTM_v55_intel21.4_2019_12US1/LOGS/CTM_LOG_255.v55_intel21.4_2019_12US1_20190522
2025-01-22 17:17:42  769.0 KiB cmaq-2019-modeling-platform/data/output_CCTM_v55_intel21.4_2019_12US1/LOGS/CTM_LOG_255.v55_intel21.4_2019_12US1_20190523
2025-01-22 17:17:42  760.2 KiB cmaq-2019-modeling-platform/data/output_CCTM_v55_intel21.4_2019_12US1/LOGS/CTM_LOG_255.v55_intel21.4_2019_12US1_20190524
2025-01-22 17:17:42  760.8 KiB cmaq-2019-modeling-platform/data/output_CCTM_v55_intel21.4_2019_12US1/LOGS/CTM_LOG_255.v55_intel21.4_2019_12US1_20190525
2025-01-22 17:17:42  760.2 KiB cmaq-2019-modeling-platform/data/output_CCTM_v55_intel21.4_2019_12US1/LOGS/CTM_LOG_255.v55_intel21.4_2019_12US1_20190526
2025-01-22 17:17:42  760.2 KiB cmaq-2019-modeling-platform/data/output_CCTM_v55_intel21.4_2019_12US1/LOGS/CTM_LOG_255.v55_intel21.4_2019_12US1_20190527
2025-01-22 17:17:42  760.5 KiB cmaq-2019-modeling-platform/data/output_CCTM_v55_intel21.4_2019_12US1/LOGS/CTM_LOG_255.v55_intel21.4_2019_12US1_20190528
2025-01-22 17:17:42  749.1 KiB cmaq-2019-modeling-platform/data/output_CCTM_v55_intel21.4_2019_12US1/LOGS/CTM_LOG_255.v55_intel21.4_2019_12US1_20190529
2025-01-22 17:17:42  760.8 KiB cmaq-2019-modeling-platform/data/output_CCTM_v55_intel21.4_2019_12US1/LOGS/CTM_LOG_255.v55_intel21.4_2019_12US1_20190530
2025-01-22 17:17:42  760.2 KiB cmaq-2019-modeling-platform/data/output_CCTM_v55_intel21.4_2019_12US1/LOGS/CTM_LOG_255.v55_intel21.4_2019_12US1_20190531
2024-11-15 12:44:16   30.9 KiB index.html
2024-11-15 13:05:34    3.8 KiB readme.html

Total Objects: 7271
   Total Size: 6.5 TiB

To query how much meterology data is in the EQUATES bucket

aws s3 --no-sign-request ls --summarize --region=us-east-1 --human-readable --recursive s3://cmas-equates/CMAQ_12US1/INPUT/2019

Output:

...
...
2024-06-05 13:25:32  707 Bytes CMAQ_12US1/INPUT/2019/met/mcip_v51_wrf_v411_noltng/12/MCIPv51_WRFv411_noltng_12US1.35L.20191227.contents.txt
2024-06-05 13:25:33    6.6 GiB CMAQ_12US1/INPUT/2019/met/mcip_v51_wrf_v411_noltng/12/MCIPv51_WRFv411_noltng_12US1.35L.20191227.tar
2024-06-05 13:28:05  707 Bytes CMAQ_12US1/INPUT/2019/met/mcip_v51_wrf_v411_noltng/12/MCIPv51_WRFv411_noltng_12US1.35L.20191228.contents.txt
2024-06-05 13:28:06    6.7 GiB CMAQ_12US1/INPUT/2019/met/mcip_v51_wrf_v411_noltng/12/MCIPv51_WRFv411_noltng_12US1.35L.20191228.tar
2024-06-05 13:31:25  707 Bytes CMAQ_12US1/INPUT/2019/met/mcip_v51_wrf_v411_noltng/12/MCIPv51_WRFv411_noltng_12US1.35L.20191229.contents.txt
2024-06-05 13:31:25    6.7 GiB CMAQ_12US1/INPUT/2019/met/mcip_v51_wrf_v411_noltng/12/MCIPv51_WRFv411_noltng_12US1.35L.20191229.tar
2024-06-05 13:32:26  707 Bytes CMAQ_12US1/INPUT/2019/met/mcip_v51_wrf_v411_noltng/12/MCIPv51_WRFv411_noltng_12US1.35L.20191230.contents.txt
2024-06-05 13:32:27    6.7 GiB CMAQ_12US1/INPUT/2019/met/mcip_v51_wrf_v411_noltng/12/MCIPv51_WRFv411_noltng_12US1.35L.20191230.tar
2024-06-05 13:36:11  707 Bytes CMAQ_12US1/INPUT/2019/met/mcip_v51_wrf_v411_noltng/12/MCIPv51_WRFv411_noltng_12US1.35L.20191231.contents.txt
2024-06-05 13:36:11    6.7 GiB CMAQ_12US1/INPUT/2019/met/mcip_v51_wrf_v411_noltng/12/MCIPv51_WRFv411_noltng_12US1.35L.20191231.tar

Total Objects: 905
   Total Size: 2.7 TiB

Command to copy files from the bucket to your local filesystem (the system where you are running the aws cli) and place it under the path that you specify for /your_local_path. To obtain the emissions, icbc, and epic files:

aws s3 --no-sign-request --region=us-east-1 cp --recursive s3://cmaq-2019-modeling-platform/ /your_local_path/

To obtain the meterology files:

aws s3 --no-sign-request --region=us-east-1 cp --recursive s3://cmas-equates/CMAQ_12US1/INPUT/2019 /your_local_path/

To obtain all input data and run scripts for a two day run use the s3 download script available here. Note, you need to edit the script to specify the local directory on your machine. To run a 2 day run starting on 20190522 - 20190523, you will need to download 123G of data using the following script:

wget https://cmaq-2019-modeling-platform.s3.amazonaws.com/cmaq-2019-modeling-platform/CMAQv53_TS/2019_12US1/scripts/s3_copy_nosign_2019_cracmm_opendata_to_local_2019.csh
##Edit lines 16 and 17 to specify the path of your local directory for the input data
./s3_copy_nosign_2019_cracmm_opendata_to_local_2019.csh

Please submit any questions or difficulties as a new issue in the CMAS Data Warehouse Category on the CMAS User Forum.
CMAS User Forum