diff --git a/README.md b/README.md new file mode 100644 index 0000000..725adeb --- /dev/null +++ b/README.md @@ -0,0 +1,109 @@ +Summary +------- + +AutoAPI is a very simple, very scalable API engine that converts flat data files into a web service. To add, edit, and remove data from the API, you just add, overwrite, or delete flat CSV files from either an s3 bucket or your local filesystem. + +Currently, AutoAPI is the core offering of the +[18F API Program](https://pages.18f.gov/api-program/), a comprehensive solution for government agencies to get up-and-running with production APIs quickly. + + +Example Instance +---------------- + +* https://autoapi.18f.gov/ +* https://autoapi2.18f.gov + +How to use: + +* To see json, you can `curl https://autoapi.18f.gov` +* You can also use an API viewer like [Postman for Chrome] (https://chrome.google.com/webstore/detail/postman/fhbjgbiflinjbdggehcddcbncdddomop?hl=en) + +Query Parameters: + +* `?page=2` +* `?[columnheader]=[value]` +* `?[columnheader1]=[value1]&[columnheader1]=[value2] (returns results that have value1 OR value2)` +* `?[columnheader1]=[value1]&[columnheader3]=[value4] (returns results that have both value1 and value2)` + + +Requirements +------------ + +In order to use autoapi, you'll need: + +* Python 3.5 +* npm + +Alternatively, if you use Docker for development and/or deployment, you don't +need anything (except Docker, of course). + + +Quickstart (without Docker) +--------------------------- + +First, let's get dependencies sorted out: + + pip3 install -r requirements.txt + npm install + +The easiest way to get started with autoapi is by serving CSV files from +your local filesystem. + +Here's a quick way to generate some sample data:: + + mkdir data_sources + cat << EOF > data_sources/my_sample_data.csv + name,color + apple,red + grapefruit,pink + EOF + +Now you can add your sample data to a local SQLite database:: + + invoke apify "data_sources/**.*" + +Finally, you can start the server with: + + invoke serve + +Now try visiting http://localhost:5000 in your browser, or poke at +the following URLs from the command-line: + + curl http://localhost:5000/my_sample_data + curl http://localhost:5000/my_sample_data/1 + curl http://localhost:5000/my_sample_data?color=pink + +**More details can be found in [instructions.md](instructions.md).** + + +Quickstart (with Docker) +------------------------ + +If you have `Docker `_ installed, you can run from a Docker +container with no further setup:: + + docker run \ + -p 5000:5000 \ + -v `pwd`/data_sources:/data_sources \ + --rm -it 18fgsa/autoapi + +Any environment variables can be set via the ``-e NAME=VALUE`` option. + +Note that this sets up your autoapi instance by directly +pulling autoapi from its +[Docker Hub image](https://hub.docker.com/r/18fgsa/autoapi/); it doesn't +even require you to clone autoapi's git repository. However, as a +result you won't be able to develop autoapi itself. + +**More detailed instructions for running with Docker are +[here](instructions.md).** + +Public domain +--------------- + +This project is in the worldwide [public domain](LICENSE.md). As stated in +[CONTRIBUTING ](CONTRIBUTING.md): + +> This project is in the public domain within the United States, and copyright and related rights in the work worldwide are waived through the [CC0 1.0 Universal public domain dedication] (https://creativecommons.org/publicdomain/zero/1.0/). + +> All contributions to this project will be released under the CC0 dedication. By submitting a pull request, you are agreeing to comply with this waiver of copyright interest. diff --git a/README.rst b/README.rst deleted file mode 100644 index 46d0f5c..0000000 --- a/README.rst +++ /dev/null @@ -1,153 +0,0 @@ -Summary -------- - -AutoAPI is a very simple, very scalable API engine that converts flat data files into a web service. To add, edit, and remove data from the API, you just add, overwrite, or delete flat CSV files from either an s3 bucket or your local filesystem. - -Currently, AutoAPI is the core offering of the `18F API Program `_, a comprehensive solution for government agencies to get up-and-running with production APIs quickly. - - -Example Instance ----------------- - -* https://autoapi.18f.gov/ -* https://autoapi2.18f.gov - -How to use: - -* To see json, you can `curl https://autoapi.18f.gov` -* You can also use an API viewer like `Postman for Chrome `_ - -Query Parameters: - -* ?page=2 -* ?[columnheader]=[value] -* ?[columnheader1]=[value1]&[columnheader1]=[value2] (returns results that have value1 OR value2) -* ?[columnheader1]=[value1]&[columnheader3]=[value4] (returns results that have both value1 and value2) - - -Requirements ------------- - -In order to use autoapi, you'll need: - -* Python 3.5 -* npm - -Alternatively, if you use Docker for development and/or deployment, you don't -need anything (except Docker, of course). - - -Quickstart ----------- - -First, let's get dependencies sorted out:: - - pip3 install -r requirements.txt - npm install - -The easiest way to get started with autoapi is by serving CSV files from -your local filesystem. - -Here's a quick way to generate some sample data:: - - mkdir data_sources - cat << EOF > data_sources/my_sample_data.csv - name,color - apple,red - grapefruit,pink - EOF - -Now you can add your sample data to a local SQLite database:: - - invoke apify "/data_sources/**.*" - -Finally, you can start the server with:: - - invoke serve - -Now try visiting http://localhost:5000 in your browser, or poke at -the following URLs from the command-line:: - - curl http://localhost:5000/my_sample_data - curl http://localhost:5000/my_sample_data/1 - curl http://localhost:5000/my_sample_data?color=pink - -More details can be found in `instructions.md `_. - - -Deployment via docker ---------------------- - -If you have `Docker `_ installed, you can run from a Docker -container with no further setup:: - - docker run \ - -p 5000:5000 \ - -v `pwd`/data_sources:/data_sources \ - --rm -it 18fgsa/autoapi - -Any environment variables can be set via the ``-e NAME=VALUE`` option. - -Note that this sets up your autoapi instance by directly -pulling autoapi from its -`Docker Hub image `_; it doesn't -even require you to clone autoapi's git repository. However, as a -result you won't be able to develop autoapi itself. - - -Development via docker ----------------------- - -If you'd like to use Docker for developing autoapi, just clone its -repository, build the docker image and start the server:: - - docker-compose build - docker-compose up - -Now you can visit your server at http://localhost:5000. - -If you want to set any environment variables, you can do so by creating -a ``.env`` file in the root directory of the repository, where each line -consists of a ``NAME=VALUE`` pair. - -If any dependencies change, such as those listed in ``requirements.txt`` -or ``package.json``, just re-run ``docker-compose build``. - -For more information on using Docker for development, see the -`18F Docker Guide `_. - - -Database Configuration ----------------------- - -By default, autoapi uses a local SQLite database. To specify a different database URI, set the ``DATABASE_URL`` environment variable. For use with Cloud Foundry, simply create and bind an RDS service; this will automatically configure the environment. - -:: - - cf create-service rds shared-psql autoapi-rds - cf bind-service autoapi autoapi-rds - -For details on RDS services available through 18F Cloud Foundry, see https://docs.cloud.gov/apps/databases/. - - -AWS Integration ---------------- - -To use AWS instead of local CSV files, you'll want to define the following -environment variables: - -* ``AUTOAPI_BUCKET`` - The bucket containing your CSV files. -* ``AWS_ACCESS_KEY_ID`` - Your AWS access key. -* ``AWS_SECRET_ACCESS_KEY`` - Your AWS secret access key. - -**autoapi** synchronizes with the S3 bucket specified in the ``AUTOAPI_BUCKET`` environment variable. On starting the API server, **autoapi** creates a subscription to the target bucket using Amazon SNS. When files are added to or deleted from the bucket, the corresponding endpoints will automatically be updated on the API. - - -Public domain ---------------- - -This project is in the worldwide `public domain `_. As stated in `CONTRIBUTING `_: - - This project is in the public domain within the United States, and copyright and related rights in the work worldwide are waived through the `CC0 1.0 Universal public domain dedication `_. - - All contributions to this project will be released under the CC0 dedication. By submitting a pull request, you are agreeing to comply with this waiver of copyright interest. diff --git a/instructions.md b/instructions.md index 6957f71..070577a 100644 --- a/instructions.md +++ b/instructions.md @@ -50,6 +50,8 @@ your files and API server will 'live'.
  • View page at IP address from docker-machine, typically http://192.168.99.100:5000
    • + To develop autoapi itself with Docker, see + these instructions. @@ -181,9 +183,9 @@ Set local environment variables for S3 Using your [S3 credentials](#s3-credentials), - export AWS_ACCESS_KEY_ID=(Access Key ID from IAM user) - export AWS_SECRET_ACCESS_KEY=(Secret Access Key from IAM user) - export AUTOAPI_BUCKET=(your s3 bucket name) + export AWS_ACCESS_KEY_ID=(AWS Access Key ID from IAM user) + export AWS_SECRET_ACCESS_KEY=(AWS Secret Access Key from IAM user) + export AUTOAPI_BUCKET=(name of s3 bucket containing CSV files)
    @@ -256,6 +258,31 @@ Set cloud.gov environment for S3 cf set-env $AWS_SECRET_ACCESS_KEY AWS_SECRET_ACCESS_KEY (Secret Access Key from IAM user) cf set-env $AUTOAPI_NAME AUTOAPI_BUCKET (your s3 bucket name) +
    + +Developing with Docker +---------------------- + +The Docker instructions above will let you use autoapi as-is, +but if you'd like to use Docker for *developing* autoapi, +just clone its +repository, build the docker image and start the server:: + + docker-compose build + docker-compose up + +Now you can visit your server at http://localhost:5000. + +If you want to set any environment variables, you can do so by creating +a ``.env`` file in the root directory of the repository, where each line +consists of a ``NAME=VALUE`` pair. + +If any dependencies change, such as those listed in ``requirements.txt`` +or ``package.json``, just re-run ``docker-compose build``. + +For more information on using Docker for development, see the +[18F Docker Guide](https://pages.18f.gov/dev-environment-standardization/virtualization/docker/). + Refreshing data =============== @@ -303,3 +330,13 @@ Optional configuration ``` export DATABASE_URL=postgresql://myusername:mypassword@localhost/apidb ``` + +Database Configuration +---------------------- + +By default, autoapi uses a local SQLite database. To specify a different database URI, set the ``DATABASE_URL`` environment variable. For use with Cloud Foundry, simply create and bind an RDS service; this will automatically configure the environment. + + cf create-service rds shared-psql autoapi-rds + cf bind-service autoapi autoapi-rds + +For details on RDS services available through 18F Cloud Foundry, see https://docs.cloud.gov/apps/databases/. diff --git a/run_in_docker.sh b/run_in_docker.sh index 929f405..237ec41 100755 --- a/run_in_docker.sh +++ b/run_in_docker.sh @@ -3,5 +3,5 @@ cd /autoapi if [ -n "$AUTOAPI_BUCKET" ]; then invoke fetch_bucket fi -invoke apify "/data_sources/**.*" +invoke apify "/data_sources/*.*" invoke serve