Access Data

Cloud Hosted Archive

Getting Started With SCEDC AWS Public Dataset

This page describes some quick examples in python to start using the SCEDC dataset.

Python Code Snippet
Docker Image
Amazon Machine Image (tutorial available)

Requirements

AWS account: sign up at https://aws.amazon.com/console/
AWS credentials: For programmatic access you need Access Keys.
boto3 and botocore: AWS SDK for Python

Python Snippet

import boto3, botocore
s3=boto3.resource('s3')
BUCKET_NAME = 'scedc-pds'
KEY='continuous_waveforms/2017/2017_180/CIGSC__BHZ___2017180.ms'
try:

       s3.Bucket(BUCKET_NAME).download_file(KEY,'CIGSC__BHZ___2017180.ms')
except botocore.exceptions.ClientError as e:
       if e.response['Error']['Code'] == "404":
            print("The object does not exist.")
       else:
            raise

Docker Example

This example shows how to run a Docker image with ObsPy and Jupyter Notebook. You can then run the python snippet and start using classes from the Obspy framework.

run from os prompt:
~ $: docker pull crleblanc/obspy-notebook
~ $: docker run -e AWS_ACCESS_KEY_ID= -e AWS_SECRET_ACCESS_KEY= -p 8888:8888 crleblanc/obspy-notebook:latest
~ $: docker exec pip install boto3

Using an Amazon Machine Image (AMI)

There is a public AMI image called scedc-demo that has a Linux OS, python, boto3 and botocore installed.

from your AWS management console, choose "EC2"
Under "Instances" choose to launch an instance. Select existing AMI named "scedc-demo" (under Community AMIs)

For a more detailed step-by-step, look at this tutorial.

References

SCEDC Cloud Scripts Scripts to download continuous waveforms, event based waveforms, and phases from the AWS public dataset.
Cactus to Clouds: Processing The SCEDC Open Data Set on AWS A conference paper (2019 SCEC Annual Meeting) by Tim Clements and Marine Denolle at Harvard University detailing their work using the SCEDC dataset
Downloading using parallel threads A more sophisticated script which may increase download speeds written by Tim Clements
Boto 3 Documentation
AWS CLI User Guide AWS Command Line Interface. An open source tool for working with AWS via command line.
AWS Getting Started Resource Center Starter page for learning about what types of AWS resources are available.