Access Data
Cloud Hosted Archive
Getting Started With SCEDC AWS Public Dataset
This page describes some quick examples in python to start using the SCEDC dataset.
Requirements
AWS account: sign up at https://aws.amazon.com/console/
AWS credentials: For programmatic access you need Access Keys.
boto3 and botocore: AWS SDK for Python
Python Snippet
import boto3, botocore
s3=boto3.resource('s3')
BUCKET_NAME = 'scedc-pds'
KEY='continuous_waveforms/2017/2017_180/CIGSC__BHZ___2017180.ms'
try:
s3.Bucket(BUCKET_NAME).download_file(KEY,'CIGSC__BHZ___2017180.ms')
except botocore.exceptions.ClientError as e:
if e.response['Error']['Code'] == "404":
print("The object does not exist.")
else:
raise
Docker Example
This example shows how to run a Docker image with ObsPy and Jupyter Notebook. You can then run the python snippet and start using classes from the Obspy framework.
run from os prompt:
~ $: docker pull crleblanc/obspy-notebook
~ $: docker run -e AWS_ACCESS_KEY_ID= -e AWS_SECRET_ACCESS_KEY= -p 8888:8888 crleblanc/obspy-notebook:latest
~ $: docker exec pip install boto3
Using an Amazon Machine Image (AMI)
There is a public AMI image called scedc-demo that has a Linux OS, python, boto3 and botocore installed.- from your AWS management console, choose "EC2"
- Under "Instances" choose to launch an instance. Select existing AMI named "scedc-demo" (under Community AMIs)
References
- SCEDC Cloud Scripts Scripts to download continuous waveforms, event based waveforms, and phases from the AWS public dataset.
- Cactus to Clouds: Processing The SCEDC Open Data Set on AWS A conference paper (2019 SCEC Annual Meeting) by Tim Clements and Marine Denolle at Harvard University detailing their work using the SCEDC dataset
- Downloading using parallel threads A more sophisticated script which may increase download speeds written by Tim Clements
- Boto 3 Documentation
- AWS CLI User Guide AWS Command Line Interface. An open source tool for working with AWS via command line.
- AWS Getting Started Resource Center Starter page for learning about what types of AWS resources are available.