near/berliner-winter
1
0
Fork 0
mirror of https://github.com/heyarne/berliner-winter.git synced 2026-05-06 19:23:39 +02:00
Code Issues Projects Releases Wiki Activity
95 commits 3 branches 0 tags 10 MiB
Commit graph

95 commits

This branch
This branch
All branches
Author SHA1 Message Date
Arne Schlüter
13791c0599 Add API for location 2015-02-07 17:23:43 +01:00
Arne Schlüter
c09a1d78a2 Use list of german nouns before geocoding 2015-02-07 14:53:27 +01:00
Arne Schlüter
9cbff4aa30 Add potentially interesting statistics 2015-02-07 14:53:09 +01:00
Arne Schlüter
430dda0cc6 Add TODO note about improvement to locator.py 2015-01-18 21:05:54 +01:00
Arne Schlüter
872d7b219d Improve query to the google maps geocoding api 2015-01-18 20:55:47 +01:00
Arne Schlüter
b229e1a6c4 Add category to the database 2015-01-18 11:56:25 +01:00
Arne Schlüter
24ffdeb1ec Add niceness and log output to the geocoding 2015-01-18 11:22:07 +01:00
Arne Schlüter
cb0175a7b7 Move geocoding to Google API 2015-01-17 19:12:09 +01:00
Arne Schlüter
13ac77b575 Improve urlencoding of location query 2015-01-17 17:48:39 +01:00
Arne Schlüter
41f2afcc12 Merge branch 'place_extraction' 
Conflicts:
	analyze.py
 2015-01-17 17:38:41 +01:00
Joshua Widmann
309cd191af Function to get longitude and latitude from certain query to OSM. 2015-01-17 17:37:18 +01:00
Arne Schlüter
492ea184bc Fix POS-tuple cleanup 2015-01-17 17:36:14 +01:00
Arne Schlüter
adb0252130 Fix bug so one potential place following another directly is also matched 2015-01-17 17:12:10 +01:00
Arne Schlüter
73dee66d16 Look at title and text separately 2015-01-17 16:27:53 +01:00
Arne Schlüter
b85c84139b Start with improving the place results 2015-01-17 15:57:51 +01:00
Arne Schlüter
8e6032be3a Start messing around with part of speech tagging 2015-01-17 15:15:51 +01:00
Arne Schlüter
c4711310ce Merge branch 'master' of github.com:aesthaddicts/OpenData 2015-01-17 14:42:22 +01:00
Joshua Widmann
4ddd3aa376 swapped '…' with '...' and removed sqlite3 import. 2015-01-17 13:58:38 +01:00
Arne Schlüter
06814ecae8 Start writing some analyzaton code 2015-01-17 11:45:52 +01:00
Arne Schlüter
a9a096396b Install nltk for named entity extraction of places later on 2015-01-17 11:43:59 +01:00
Arne Schlüter
00f0643fff Add redis and rq as a dependency for incident analyzation 2015-01-17 11:42:38 +01:00
Arne Schlüter
7c4cf8f9f0 Clean up code, clarify and remove an unnecessary try-except-block 2014-12-11 00:29:08 +01:00
Arne Schlüter
98d1e21a90 Remove field 'addtional_place' because it can't be reliably parsed 2014-12-11 00:07:48 +01:00
Arne Schlüter
c1ac5e5ed4 Use peewee as model and rewrite the code 2014-12-10 23:56:27 +01:00
Arne Schlüter
0e095dbb63 Don't forget to close the connection 2014-12-08 20:40:13 +01:00
Arne Schlüter
36df116ed0 Crawl all pages and insert them into the database 2014-12-08 18:26:25 +01:00
Arne Schlüter
5306e6dab4 Merge branch 'dev-arne' 
Conflicts:
	scraper/scraper.py
 2014-12-08 16:20:08 +01:00
Arne Schlüter
55a599a47b Write insertion logic for articles 2014-12-08 16:19:18 +01:00
Joshua Widmann
c872c8e39d renamed article attributes according to database fields 2014-12-08 15:59:33 +01:00
Arne Schlüter
e018aead0e Add database setup code 2014-12-08 15:40:31 +01:00
Joshua Widmann
9ae38d3279 scraping all articles 2014-12-08 15:38:58 +01:00
Arne Schlüter
58f8e65669 Simplify code by removing :nth-of-type and accessing the list directly 2014-12-08 14:46:26 +01:00
Arne Schlüter
66b68b269d Normalize \r\n to \n and use date objects instead of the original date string 2014-12-08 14:40:05 +01:00
Arne Schlüter
7fe08227bb Add more info to the README 2014-11-17 18:32:05 +01:00
Arne Schlüter
0744167bde Implement basic article scraping 2014-11-17 18:25:46 +01:00
Arne Schlüter
2b83417f8e Add python-typic files and folders to .gitignore 2014-11-17 18:25:28 +01:00
Arne Schlüter
5fc4b7b095 Separate into different files 2014-11-17 14:37:41 +01:00
Arne Schlüter
c44c5a35b8 Add map overview of berlin with toner-lite map tiles 2014-11-17 14:36:03 +01:00
Arne Schlüter
b7f571b20f Set up basic HTML structure with necessary style rules 2014-11-17 14:26:33 +01:00
Arne Schlüter
c12c8b6dbe Add .editorconfig for code formatting 2014-11-11 07:43:19 +01:00
Arne Schlüter
fc54df1ca8 Update project description and readme 2014-11-10 15:32:37 +01:00
Arne Schlüter
d09c324c8d Remove data of bundeswahlleiter.de 2014-11-10 15:29:32 +01:00
Arne Schlüter
8323a82305 Add leaflet as our map service 2014-11-10 15:28:56 +01:00
Arne Schlüter
0f4b9d8805 Convert csv to UTF-8 2014-11-10 14:45:28 +01:00
Arne Schlüter
dbc50d8e58 Initial commit 2014-11-10 14:41:21 +01:00