Erik Schwartz

Assuming you have a shapefile

shp2pgsql -s 4326 VotingPrecinct/VotingPrecinct.shp | psql -d postgis_demo

csvsql --db postgresql:///postgis_demo --insert --table ce_calls ce_calls_gecoded.csv

update column names manually since the camelcase columns are not usable in postgres (today for me anyway)

head -n 1 ce_calls_geocoded_bad_headers.csv > ce_calls_gecoded.csv
vim ce_calls_geocoded.csv # fix the headings
# tail all but the header line
tail -n +2 ce_calls_geocoded_bad_headers.csv >> ce_calls_gecoded.csv

Error message

You don't appear to have the necessary database backend installed for connection string you're trying to use. Available backends include:

Postgresql:	pip install psycopg2
MySQL:		pip install MySQL-python

For details on connection strings and other backends, please see the SQLAlchemy documentation on dialects at:

http://www.sqlalchemy.org/docs/dialects/

pip install psycopg2

in qgis: Layer > Add PostGIS layers

Problem: csvsql doesn’t insert lat long as point!

Add point column to ce_calls table

SELECT AddGeometryColumn ('public','ce_calls','geom',4326,'POINT',2);
UPDATE ce_calls SET geom = ST_SetSRID(ST_MakePoint(geo_longitude, geo_latitude), 4326);

Update SRID of voting precinct table

SELECT UpdateGeometrySRID('votingprecinct','geom',4326);

join on point in voting precinct

select objectid, c.* from votingprecinct v join ce_calls c ON (st_contains(v.geom, c.geom));

select counts against years

in a file named ce_calls.sql

copy
(select objectid as id,
  sum(case when extract(year from dateopened) = 2011 then 1 else 0 end) as y_2011,
  sum(case when extract(year from dateopened) = 2012 then 1 else 0 end) as y_2012,
  sum(case when extract(year from dateopened) = 2013 then 1 else 0 end) as y_2013,
  sum(case when extract(year from dateopened) = 2014 then 1 else 0 end) as y_2014
  from votingprecinct v inner join ce_calls c ON(st_contains(v.geom, c.geom) AND casetype = 'Housing')
  group by objectid
  order by objectid
) TO STDOUT WITH CSV HEADER;

run the query from command line, and export to stdout as csv

cat ce_calls.sql | xargs -0 psql -d postgis_demo -c > assets/data/metric/ce-housing.csv

Running OSX Mavericks

Following the official qgis install guide

Installs everything to: /Library/Frameworks/

http://www.kyngchaos.com/files/software/frameworks/GEOS_Framework-3.4.2-3.dmg
http://www.kyngchaos.com/files/software/frameworks/PROJ_Framework-4.8.0-1.dmg
http://www.kyngchaos.com/files/software/frameworks/UnixImageIO_Framework-1.5.0.dmg
http://www.kyngchaos.com/files/software/frameworks/SQLite3_Framework-3.7.17-2.dmg
http://www.kyngchaos.com/files/software/frameworks/GDAL_Framework-1.11.0-4.dmg
# install NumPy and GDal complete framework
http://www.kyngchaos.com/files/software/python/matplotlib-1.3.1-2.dmg
http://www.kyngchaos.com/files/software/qgis/QGIS-2.4.0-1.dmg

brew install postgresql
brew install postgis
createdb postgis_demo
psql -d postgis_demo -c "CREATE EXTENSION postgis;"

We had a fantastic event last week at Code for America called BETA, a so called “civic show and tell”. It was recorded for posterity, the only snag being that we didn’t record all of the magnificent slides that people produced. The good news is that we can hack together a way to embed them in the video along with the lovely presenters.

Problem Statement

Shucks, I have video of presentors but not their slides.

Solution

Follow along to the presentor video
Record the slide transitions in Keynote
Export the perfectly-timed Keynote slides as a video
Embed the slides video in the presentor video like a boss
…
Profit

Record slide transitions in Keynote (version 6)

Open the presentor video and get ready to play it
In Keynote menu Play > Record Slideshow...
Start the presentor video so that you can listen along
Back to Keynote, click the mic to mute audio input. Click record.
Transition the slides as you listen to the presentation
File > Export To > Quicktime

Embed slides video into presentation video in iMovie (version 10)

Create event, create new video (called Lexingteam in image below) and import your presentor video
Drag the slides video above the presentor video
Move the slides video left/right until the transitions line up with the presentors

Adjust the slides video

Click the ‘Adjust’ menu in the upper right
Select the slides video clip in the video area below
Set to ‘picture in picture’ and set the dissolve time to 0.0
Drag and resize the slides video window to your liking (you can also change ‘dissolve’ to ‘swap’ to make your slides the main picture and your presentation video the smaller picture)

Video editing tips from a n00b

You can click and hold on a part of the slides video to copy and repeat a section. This is helpful to make the intro slide longer.
If you have several video or picture sections above the ‘main’ video (for instance, if you copied the beginning slide to cover the beginning of the presentation), you can copy picture-in-picture adjustments like resizing the slides video. Adjust the video clip, Edit > Copy, select all of the clips and Edit > Paste Adjustments > All.

Bash: sort ruby files by number of lines (ascending)

find . -name '*.rb' -exec wc -l {} \; | sort
I used this to find a few hotspots in calagator

...
./spec/controllers/venues_controller_spec.rb
./spec/models/source_parser_ical_spec.rb
./spec/models/venue_spec.rb
./spec/controllers/sources_controller_spec.rb
./app/models/event.rb
./spec/controllers/events_controller_spec.rb
./spec/models/event_spec.rb

Heroku: backup postgres

heroku addons:add pgbackups:auto-month
heroku pgbackups:capture # backup right now!
heroku pgbackups # show the backups
load to local env curl -o latest.sql `heroku pgbackups:url
pg_restore --verbose --clean --no-acl --no-owner -h localhost -U myuser -d mydb latest.dump (via heroku)

Bash: save command you’ve typed without running it

Ctrl+a to jump to beginning of line and comment it out :) $ #echo 'this is a command for later'

Zsh: save command you’ve typed without running it

“Ctrl+Y will paste the last item you cut (with Ctrl+U, Ctrl+K, Ctrl+W, etc.)”

Sed: remove leading chars

echo 'this is a new new thing' | sed 's/^this //' => “is a new new thing”
echo 'this is a new new thing' | sed 's/.\{5\}//' => “is a new new thing”
echo ' remove leading whitespace' | sed 's/[ \t]*//' => “remove leading whitespace”

Kimono: retrieving crawled pages with the source url included

Add &kimbypage=1 to the query string

"results": [
    {
      "page": 1,
      "url": "http://www.lexpublib.org/event/friends-of-library-booksale-uk-chandler-medical-center",
      "collection1": [
        {
          "title": "Friends of the Library Booksale: UK Chandler Medical Center",
          "venue": "Other Location",
          "access_notes": "See Description",
          "description": "Friends of the Library will be have books for sale at the UK Chandler Medical Center.",
          "start_time": "Wednesday, May 21st - 8:00 AM"
        }
      ]
    },

Ruby: regex capture syntax

1.9.3-p484 :002 > 'xxfindmexx'.gsub(/xx(findme)xx/, '\1')
 => "findme"
# note that double quotes around \1 interprets it as a unicode escape
1.9.3-p484 :003 > 'xxfindmexx'.gsub(/xx(findme)xx/, "\1")
 => "\u0001"

Markdown: fix the fragment fn-infer behavior

Now it drops in a named link: <a name="do-some-cool-thing"></a> [do some cool thing](/2014-05-23-environment-improvements-3#do-some-cool-thing) => <a name="do-some-cool-thing"></a>[do some cool thing](#do-some-cool-thing)
It’s borderline unreadable as markdown syntax but hey, what can you do?

Live-ish blogging Solid Con!

Blue City

Hardware infrastructure to pipe point of observation data to the platform
Aggregate various atmospheric city data through resident sensors on phones and other devices
BlueData platform to aggregate multiple data sources
saha@bluecityllc.com

Data Vehicles for Open Mobility in Cities

Kabal Farah (MIT Media Lab)
City is a collection of desire paths
Map connections being created in space via mobile
First distruptive technology
- the car
- 15,000,000 sold in first wave
Self driving cars
- You experience city in new way
  - Office space
  - Familiy Space
- BAEDE-KAR: 8-track faux-gps
- Ford releasing Open XC API
- The car becomes aware of its technology
  - Starts to approach automation of high-tech warehouses of today
Build up preferences, experiences
- City is able to predict what is next for you, for everyone
- Creepy?

IoT and Restful Hardware

Alden Hart - Synthetos
Motion control sensors, motor controllers for CnC machines.
TinyG montion controller. Takes Gcode directly. 6-way motion
They got the electronics working but people didn’t like integrating with unstructured text protocol
They wanted to eliminate begel bone or raspberry pi as a base requirement, only as enchancement
They looked to REST and Fielding’s work where state resides on server
Need well defined resource model that describes machine very well
They replaced server with their chip
They use JSON support directly on hardware
Run JSON right over traces
Wrote their own parser, strict and relaxed mode
7 microseconds strtof() < 7uSec on 84 MHz ARM Cortex M3
JSON subset
- 7bit utf
- no linefeeds
- Flat namespace, don’t go wild nesting
POST means execute
cheats
- No request headers. null means get
- They’re all point to point
- Will execute Gcode directly, doesn’t have to be wrapped in JSON
Use footers to verify integrity
Have event reporting to communicate state deltas
Don’t innovate. Copy
- Use $ as root (from JSONpath)
- @ prefix for adressing, binding, routing
- # topic prefix
- ! priority. Jump the queue
Why not protocol buffers?
- To support late binding of devices
- telehash has binary payloads which is cool. Another possibility

Smart Design: Attention to microinteractions to go from prototype to product

Water watcher
- Contextual feedback about usage.
- Immediate indication
Difference between prototype and product
- prototype: demo a feature
- product is separated from prototype by many microinteractions
  - change a setting, log in, error recovery, ….
  - consider apple with one button OBE versus some thing you have to debug immediately
  - toaster with the ‘a little bit more’ button. They know their UX.
  - product is prototype after you remove all the ‘pebbles in your shoe’
  - Slate is upselling improved microinteractions (comments in-line and reduced pagination)
Anatomy of a microinteraction. Be deliberate about:
- trigger
  - Nest smoke detector glows and beeps so you can find the one with batteries low
- rules
  - OSX fans turn down when you’re dictating
- feedback
  - Ford shows leaves next to MPG which cause behavior change in drivers
- Loops and Modes

Announcing a Product Before It’s Ready - Stories From the Frontline

Jason Johnson, August Home Inc
Using (Near field communication) NFC for doorlocks
Bluetooth 4.0
Yves Behar’s design
ODM (original design manufacturer): someone with a product you can build on

The Collision of Privacy, Regulation, and Physical Computing

… joined late

Chris Clearfield (System Logic)
On-star
- Asked about being hacked and GM declared it ‘Read-Only’. “Until someone figures out otherwise”
Mobile radios
- Mobile Devices - Base band processors in common consumer phones
- History of being hacked: full access to filesystem proof-of-concept
Data and privacy within stores
- Use wifi MAC addresses to track people consistently
- “Conversnitch” - plug-in lightbulb replacement with mic that pushes to Mechanical Turk and Twitter
Private companies collection
- If data breach or legally compelled to give over your data, it’s hugely telling of your life
Charles Perrow on collisions between tech and privacy
- Organizations struggle to manage high risk systems
  - One failure leads to more: such tight coupling
  - So many org and people boundaries
    - Mat Honan example of confidence escalation
  - Two-factor accounts help with caveats
Hard to control in the expanding network
- Cars
- Fridges
- TVs
- Phones
- Meta-data up for grabs
Solutions
- Cultivate sensitivity. Be intentional
- Coupling yields problems
- Take organizational approach: designate sceptics on your design team

Testing across weird interfaces and devices

Jason Huggins (Sauce Labs Inc) @hugs
Nike fuel band needs to shake to test the app that captures its date
iPhone app to open your zipcar
- you need a phone and a car
You can pay zillions to build simulators that might be wrong
@hugs has developed a robot tester: Tapster
- (Low to High level testing pyramid) Unit -> Service -> Simulators -> Real World
- He shipped 70ish to phone manufacturers, other companies
To create your own tester bot
- Don’t spend 50k for a human hand robot
- Write a robot, motors, rubber pads, etc
Manufacturers prefer reality
Everthing on earth needs testing. Either you do it or your users do.
Tapster in action

Disrupting Space: Planet Labs From Garage to Orbit

Chris Boshuizen (Planet Labs)
31 Satellites in orbit
Map the earth every day
Bring awareness to earth through information they generate
Rapid feedback for the planet
Blue Marble Pic (Apollo 17 on way to moon in 1972)
- Important symbol for green movement
No space startups in last few decades
PhoneSat. When Chris was at NASA
- Off the shelf Google Nexus 1 with extra batteries
- Design a package to withstand space
- Crowdsourced retrieval of picture from space to HAM radio community
- Demo’d idea but phone was limited
  - No peripheral connectivity
- Out of this came Planet labs to create better satellites
Planet Labs
- Bootstrapped for a while in Cupertino
- Hard to get rocket in space
  - Space launches slipped 18 and 12 months each
- They got their sats launched from the space station

Erik Schwartz

Explorer, weirdo, developer of wonderful things.

Stay curious.