Initial commit

CHECKS

@@ -0,0 +1 @@
+/   Datasette

Procfile

@@ -0,0 +1 @@
+web: datasette -i dbs/*.db -m metadata.yml -h 0.0.0.0 -p $PORT --cors --setting default_cache_ttl 1800

README.org

@@ -0,0 +1,54 @@
+* datasette
+
+This repository contians everything you need to deploy SQLite databases to dokku via https://datasette.io/.
+
+- All files matching `db/*.db` will get exposed in the interface. Simply add your new database if all you want to do is make it accessible through the web.
+- You can edit `metadata.yml` to add descriptions, link to sources and add predefined queries.
+- The `requirements.txt` contains additional packages such as plugins. Please add them manually and without a version string because python package management is kind of messy.
+- The `CHECKS` file contains a URL and an expected string that is returned in this URLs HTTP response. This is added as a post-deployment sanity check and you shouldn't need to change it.
+- Optional: If anything should happen after deployment, edit the `bin/post_compile` script. You can use this to fetch data from other sources for example.
+
+The databases are assumed to be immutable or read-only. This allows us to use efficient caching by configuring nginx as a caching reverse proxy and serving content a static cache. This effectively that queries often only have to be run the first time after the database has changed and are afterwards served from a file-system cache.
+
+** Database Setup
+
+This section aims to contain all information needed to convert data from their respective source files to SQLite databases. This should make the updating process easier when the sources change
+
+*** Use snake_case for all file names
+
+#+begin_src bash
+cd sources
+for file in $(fd --type f); do
+    mv $file $(basename $file | sed 's/-/_/g')
+done
+#+end_src
+
+#+RESULTS:
+
+*** Cumulative CO2
+
+Source: https://ourworldindata.org/grapher/cumulative-co-emissions
+
+#+begin_src bash :results output replace
+csvs-to-sqlite \
+    --shape 'Entity:entity(text),Code:code(text),Year:year(integer),"Cumulative CO2 emissions":cumulative_co2_emissions(real)' \
+    -c entity -c code -i year \
+    sources/cumulative_co2_emissions.csv  dbs/climate.db
+#+end_src
+
+#+RESULTS:
+: extract_columns=('entity', 'code')
+: Loaded 0 dataframes
+: Added 1 CSV file to dbs/climate.db
+
+** Pitalls
+
+*** My Queries are slow, what do I do?
+
+Make sure SQlite uses the correct indices. You can debug this by writing
+
+#+begin_src sql
+EXPLAIN QUERY PLAN SELECT …
+#+end_src
+
+and then continuing your `SELECT` query like you normally would.

bin/post_compile

@@ -0,0 +1,2 @@
+# uncomment the following line to fetch the external sqlite database:
+# wget https://fivethirtyeight.datasettes.com/polls.db

metadata.yml

@@ -0,0 +1,9 @@
+databases:
+  climate:
+    tables:
+      cumulative_co2_emissions:
+        source: "Our World In Data"
+        source_url: "https://ourworldindata.org/grapher/cumulative-co-emissions"
+        license: "CC-BY 4.0"
+        units:
+          cumulative_co2_emissions: "tonnes"

requirements.txt

@@ -0,0 +1,3 @@
+datasette
+datasette-vega
+datasette-copyable