rails israel 2013
DESCRIPTION
Active Record 4.0 includes all sorts of exciting support for PostgreSQL! In this presentation, I show many of these improvements, and discuss why these are important for Web developers. If you haven't yet adopted PostgreSQL, now might be a great time and chance to do so.TRANSCRIPT
Integrating PostgreSQLReuven M. Lerner • [email protected]
Rails Israel Conference 2013October 9th, 2013 • Tel Aviv
1
Who am I?
• Web developer since 1993
• Linux Journal columnist since 1996
• PostgreSQL user since at least 1999
• Ruby/Rails developer, teacher since 2005
• PhD candidate in learning sciences at Northwestern University
2
But before all of that...I was a kid.
3
(Yes, a geeky kid.)
But before all of that...I was a kid.
3
4
5
6
7
Databases?!? Who needs them?
8
Er, I did. And do.
• Whadaya know? Databases are useful!
• They offer an abstraction layer for our data
• I don’t care how it is stored, so long as the data is safe, and I can get it back
9
What is a database?
Database
Store data confidently
Retrieve data flexibly
10
Relational databases
• Client-server
• All data is stored in two-dimensional tables
• Client-server communication is in SQL
• SQL is standard. Each database implements a superset of a subset of that standard.
11
“Impedance mismatch”
• You basically couldn’t find two languages that are more different from one another than Ruby and SQL:
• Ruby — object oriented, imperative
• SQL — table oriented, declarative
• Mixing them is ugly, as we all know
12
ActiveRecord!
• ActiveRecord is the default ORM (object-relational manager) in Rails
• ActiveRecord translates our Ruby method calls to SQL queries
• Query results are turned into Ruby objects and collections
13
The good news...
• ActiveRecord papers over the differences between databases!
14
... and the bad news
• ActiveRecord papers over the differences between databases!
15
The really bad news
• This means we’re:
• Writing extra code
• Slowing down our apps
• Putting logic in the wrong place
• Database logic shouldn’t be in your app (just like SQL shouldn’t be in your HTML)
16
PostgreSQL
17
PostgreSQL
17
PostgreSQL
17
PostgreSQL
17
PostgreSQL
17
PostgreSQL
17
PostgreSQL
• Very fast, very scalable.
• Amazingly flexible, easily extensible.
• Rock-solid — no crashes, corruption, major security issues for years
• Ridiculously easy administration
• It also happens to be free (MIT/BSD)
18
Brief history
• Ingres (Stonebreaker, Berkeley)
• Postgres (Stonebreaker, Berkeley)
• PostgreSQL project = Postgres + SQL
• About one major release per year
• Latest and greatest is 9.3
19
Features!
• MVCC
• Custom data types
• CTEs
• Functional, partial indexes
• Full-text search
• Server-side functions
20
Rails 4 to the rescue!
• As of Rails 4, many PostgreSQL features are supported, out of the box, by ActiveRecord
• The best of both worlds: Amazing database features, and we don’t have to use SQL!
21
But wait!
• What if I want to switch to another database?
• Won’t I lose the platform independence?
• Yes, you will.
• Fortunately, this will almost certainly not happen.
• And if it does, it’ll be super-painful anyway.
22
Wait again!
• This means that I can’t use SQLite in development, and PostgreSQL on the server.
• Yes, that’s right.
• You should use the same database in development and production. Please.
23
Data types
• Databases are like programming languages: The right choice of data type can have huge consequences in speed, size, and flexibility
• PostgreSQL has some special ones that can really come in handy!
24
Network types
• PostgreSQL supports INET, CIDR, and MACADDR types
• If you’re ever storing network addresses, you should use these, not a text type!
• Savings in space (and time), and faster comparisons (numeric vs. textual)
25
CREATE TABLE Logins (
id SERIAL PRIMARY KEY,
local_ip INET NOT NULL,
remote_ip INET NOT NULL);
INSERT INTO Logins (local_ip, remote_ip) values ('192.168.1.1', '10.1.2.3');
reuven=# INSERT INTO Logins (local_ip, remote_ip) values ('999.999.999.1000', '10.1.2.3');
ERROR: invalid input syntax for type inet: "999.999.999.1000"
LINE 1: INSERT INTO Logins (local_ip, remote_ip) values ('999.999.99...
26
Rails 4 migrations$ rails g model login local_ip:inet remote_ip:inet
class CreateLogins < ActiveRecord::Migration
def change
create_table :logins do |t|
t.inet :local_ip, nulls: :false
t.inet :remote_ip, nulls: :false
t.timestamps
end
end
end
27
Rails 4 appLogin.create!(local_ip:'192.168.1.1', remote_ip:'10.1.2.3')
Login.create!(local_ip:'192.168.1.1000', remote_ip:'10.1.2.3')
IPAddr::InvalidAddressError: invalid address
i = Login.first.local_ip
i.class
=> IPAddr
i.ipv4?
=> true
i.ipv6?
=> false
28
Managing a network?
• We also have CIDR and MAC addresses!
Netconfig.create!(net: '192.168.1.1/255.255.255.0')
29
UUIDs
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
CREATE TABLE evil_companies (
id UUID NOT NULL PRIMARY KEY DEFAULT uuid_generate_v1(),
name TEXT);
INSERT INTO evil_companies (name) VALUES ('HOT Internet');
INSERT INTO evil_companies (name) VALUES ('HOT TV');
INSERT INTO evil_companies (name) VALUES ('HOT Telephone');
INSERT INTO evil_companies (name) VALUES ('Voldemort, Ltd.');
30
Wait... extensions?
• Yes, PostgreSQL has extensions!
• Download and install them from PGXN
• (Think of them as gems for PostgreSQL)
• Some, like uuid-ossp, come by default
• Activate them with CREATE EXTENSION
31
32
With this in place...
SELECT * FROM evil_companies ;
┌──────────────────────────────────────┬─────────────────┐
│ id │ name │
├──────────────────────────────────────┼─────────────────┤
│ da7a333a-3069-11e3-b4fd-28cfe91f81e7 │ HOT Internet │
│ dc480be2-3069-11e3-a0f4-28cfe91f81e7 │ HOT TV │
│ ddadabfe-3069-11e3-af1d-28cfe91f81e7 │ HOT Telephone │
│ 4c33a128-306a-11e3-8000-28cfe91f81e7 │ Voldemort, Ltd. │
└──────────────────────────────────────┴─────────────────┘
33
Migrations
# Requires some manual tinkering
class CreateEvilCompanies < ActiveRecord::Migration
def change
enable_extension "uuid-ossp"
create_table :evil_companies, id: :uuid do |t|
t.text :name
t.timestamps
end
end
end
34
Migrations
# Requires some manual tinkering
class CreateEvilCompanies < ActiveRecord::Migration
def change
enable_extension "uuid-ossp"
create_table :evil_companies, id: :uuid do |t|
t.text :name
t.timestamps
end
end
end
34
In the app
EvilCompany.create!(name: 'HOT Internet')
EvilCompany.first.id
=> "07d3f537-c008-4006-9f2b-0592c7df3ebf"
35
JSON
• Remember the “impedance mismatch” between objects and databases?
• Now we have another one — we get data in JSON, turn it into database objects, and then turn it back into JSON
• But now PostgreSQL can store, retrieve, and query JSON!
36
CREATE TABLE bad_service (
id SERIAL,
info JSON
);
INSERT INTO bad_service (info)
values ('{"company":"HOT Internet", "badness":9}');
INSERT INTO bad_service (info) VALUES ('la la la');
ERROR: invalid input syntax for type json
LINE 1: insert into bad_service (info) values ('la la la');
^
DETAIL: Token "la" is invalid.
CONTEXT: JSON data, line 1: la...
37
JSON operators!
• As of PostgreSQL 9.3, we get operators
• Work with JSON, much like XML:SELECT info#>'{company}' from bad_service;┌────────────────┐│ ?column? │├────────────────┤│ "HOT Internet" │└────────────────┘
38
Migration
$ rails g model bad_service info:json
class CreateBadServices < ActiveRecord::Migration
def change
create_table :bad_services do |t|
t.json :info
t.timestamps
end
end
end
39
App
BadService.create!(info: '{"company":"HOT Internet"}')
b = BadService.first
b.info.class
=> Hash
b.info['company']
=> "HOT Internet"
40
Ranges
CREATE TABLE forecast (
forecast_for DATE,
hi_lo int4range);
INSERT INTO forecast (forecast_for, hi_lo) VALUES ('8-oct-2013', '[19,29]');
INSERT INTO forecast (forecast_for, hi_lo) VALUES ('9-oct-2013', '[20,32]');
INSERT INTO forecast (forecast_for, hi_lo) VALUES ('10-oct-2013', '[19,28]');
41
SELECT * FROM forecast ;
┌──────────────┬─────────┐
│ forecast_for │ hi_lo │
├──────────────┼─────────┤
│ 2013-10-08 │ [19,30) │
│ 2013-10-09 │ [20,33) │
│ 2013-10-10 │ [19,29) │
└──────────────┴─────────┘
SELECT * FROM forecast WHERE hi_lo @> 19;
┌──────────────┬─────────┐
│ forecast_for │ hi_lo │
├──────────────┼─────────┤
│ 2013-10-08 │ [19,30) │
│ 2013-10-10 │ [19,29) │
└──────────────┴─────────┘
42
Migration$ model forecast forecast_for:date hi_lo:int4range
class CreateForecasts < ActiveRecord::Migration
def change
create_table :forecasts do |t|
t.date :forecast_for
t.int4range :hi_lo
t.timestamps
end
end
end
43
Ranges in apps
Forecast.create!(forecast_for: Time.now, hi_lo: 20..30)
Forecast.first.hi_lo
=> 20...31
Forecast.first.hi_lo.class
Range
44
Arrays!
CREATE TABLE posts (
body text,
tags text[]
);
INSERT INTO posts (body, tags) VALUES ('Hello, world', '{intro, hello}');
INSERT INTO posts (body, tags) VALUES ('Hello again', '{DRY, boring, hello}');
45
SELECT * FROM posts;
┌──────────────┬────────────────────┐
│ body │ tags │
├──────────────┼────────────────────┤
│ Hello, world │ {intro,hello} │
│ Hello again │ {DRY,boring,hello} │
└──────────────┴────────────────────┘
SELECT * from posts where 'intro' = ANY(tags);
┌──────────────┬───────────────┐
│ body │ tags │
├──────────────┼───────────────┤
│ Hello, world │ {intro,hello} │
└──────────────┴───────────────┘
46
Arrays in migrations$ rails g model post body:text tags:text
class CreatePosts < ActiveRecord::Migration
def change
create_table :posts do |t|
t.text :body
t.text :tags, array: true
t.timestamps
end
end
end
47
Arrays in your app
Post.create!(body: 'First post!', tags: %w(first second third))
Post.first.tags
=> ["first", "second", "third"]
48
Normalization = DRY
• Everyone loves to talk about tagging as a great example of PostgreSQL arrays
• Um... don’t forget about normalization, the DRY of database design
• I know, it’s not cool to talk about it in the Age of NoSQL. But it really does work.
49
Premature denormalization is the
root of all database evil.
50
Hstore
• A key-value storage system in PostgreSQL!
• Any column can be defined as hstore
• Then you can query it — and you get a hash back!
• It’s sort of like Redis or memcached, but integrated into (and backed by) Postgres!
51
PostgreSQL
CREATE EXTENSION hstore;
CREATE TABLE posts2 (
body text,
tags hstore
);
INSERT INTO posts2 (body, tags)
VALUES ('Hello, there', '"hello" => 2, "boring" => 10');
INSERT INTO posts2 (body, tags)
values ('Hello again', '"DRY" => 5,
"hello" => 2, "boring" => 3');
52
SELECT * from posts2 where defined(tags, 'hello');
┌──────────────┬─────────────────────────────────────────┐
│ body │ tags │
├──────────────┼─────────────────────────────────────────┤
│ Hello, there │ "hello"=>"2", "boring"=>"10" │
│ Hello again │ "DRY"=>"5", "hello"=>"2", "boring"=>"3" │
└──────────────┴─────────────────────────────────────────┘
(2 rows)
SELECT tags -> 'boring' from posts2 ;
┌──────────┐
│ ?column? │
├──────────┤
│ 10 │
│ 3 │
└──────────┘
(2 rows)
53
And also...
SELECT * FROM posts2
WHERE (tags -> 'boring')::integer > 5;
┌──────────────┬──────────────────────────────┐
│ body │ tags │
├──────────────┼──────────────────────────────┤
│ Hello, there │ "hello"=>"2", "boring"=>"10" │
└──────────────┴──────────────────────────────┘
54
Migration$ rails g model newpost body:text tags:hstore
class CreateNewposts < ActiveRecord::Migration
def change
enable_extension "hstore"
create_table :newposts do |t|
t.text :body
t.hstore :tags
t.timestamps
end
end
end
55
In your app
Newpost.create!(:body => 'foo', :tags => {a:1, b:2})
Newpost.first.tags
=> {"a"=>"1", "b"=>"2"} # hash!
Newpost.first.tags['a']
=> "1" # now a string!
Newpost.first.tags[:a]
=> nil # not WithIndifferentAccess!
56
Indexes
• PostgreSQL offers different index types
• Specify these in your migrations
• Partial indexes (with a WHERE clause)
• Functional indexes (apply a function!)
• Full-text indexes (many languages)
• Geospatial indexes (install PostGIS!)
57
Some things are still missing
• “The Active Record way claims that intelligence belongs in your models, not in the database. As such, features such as triggers or foreign key constraints, which push some of that intelligence back into the database, are not heavily used.”
• — Rails Guide, ActiveRecord Migrations
58
Sigh.
59
60
Your data are your crown jewels!
61
Summary
• Rails good! PostgreSQL good!
• Rails + PostgreSQL = Doubleplus good!
• Rails 4 + PostgreSQL = Secure storage, flexible retrieval, and a lot of geeky DB fun!
62
Thanks!(Any questions?)
[email protected]://www.lerner.co.il/
@reuvenmlerner
054-496-8405“reuvenlerner” on Skype
63