Skip to content

Commit ef5393b

Browse files
author
Marc TEYSSIER
committed
Merge pull request #4 from moufmouf/2.0
Adding MagicJoin feature!
2 parents f738a47 + b788799 commit ef5393b

35 files changed

+1418
-94
lines changed

‎README.md

Lines changed: 77 additions & 64 deletions
Original file line numberDiff line numberDiff line change
@@ -7,13 +7,33 @@
77
What is Magic-query?
88
====================
99

10-
Magic-query is a PHP library that helps you work with complex queries that require
11-
a variable number of parameters.
10+
Magic-query is a PHP library that helps you work with complex SQL queries.
1211

13-
How does it work?
14-
-----------------
12+
It comes with 2 great features:
1513

16-
Easy! You write the query with all possible parameters.
14+
- [it helps you work with that require a variable number of parameters.](#parameters)
15+
- [**MagicJoin**: it writes JOINs for you!](#joins)
16+
17+
Installation
18+
------------
19+
20+
Simply use the composer package:
21+
22+
```json
23+
{
24+
"require": {
25+
"mouf/magic-query": "~1.0"
26+
},
27+
"minimum-stability": "dev",
28+
"prefer-stable": true
29+
}
30+
```
31+
32+
<a name="parameters"></a>
33+
Automatically discard unused parameters
34+
---------------------------------------
35+
36+
Just write the query with all possible parameters.
1737

1838
```php
1939
use Mouf\Database\MagicQuery;
@@ -34,84 +54,62 @@ $result2 = $magicQuery->build($sql, []);
3454
// The whole WHERE condition disappeared because it is not needed anymore!
3555
```
3656

37-
Installation
38-
------------
57+
Curious to know how this work? <a class="btn btn-primary" href="doc/discard_unused_parameters.md">Check out the complete guide!</a>
3958

40-
Simply use the composer package:
59+
<a name="joins"></a>
60+
Automatically guess JOINs with MagicJoin!
61+
-----------------------------------------
4162

42-
```json
43-
{
44-
"require": {
45-
"mouf/magic-query": "~1.0"
46-
},
47-
"minimum-stability": "dev",
48-
"prefer-stable": true
49-
}
50-
```
63+
Fed up of writing joins in SQL? Let MagicQuery do the work for you!
5164

52-
Why should I care?
53-
------------------
65+
Seriously? Yes! All you have to do is:
5466

55-
Because it is **the most efficient way to deal with queries that can have a variable number of parameters**!
56-
Think about a typical datagrid with a bunch of filter (for instance a list of products filtered by name, company, price, ...).
57-
If you have the very common idea to generate the SQL query using no PHP library, your code will look like this:
67+
- Pass a **[Doctrine DBAL connection](http://docs.doctrine-project.org/projects/doctrine-dbal/en/latest/)** to MagicQuery's constructor. MagicQuery will analyze your schema.
68+
- In your SQL query, replace the tables with `magicjoin(start_table)`
69+
- For each column of your query, use the complete name ([table_name].[column_name] instead of [column_name] alone)
5870

59-
###Without Magic-query
60-
<div class="alert"><strong>You should not do this!</strong></div>
71+
Let's assume your database schema is:
6172

62-
```php
63-
// People usually write queries like this:
64-
$sql = "SELECT * FROM products p JOIN companies c ON p.company_id = c.id WHERE 1=1 ";
65-
// They keep testing for parameters, and concatenating strings....
66-
if (isset($params['name'])) {
67-
$sql .= "AND (p.name LIKE '".addslashes($params['name'])."%' OR p.altname LIKE '".addslashes($params['name'])."%')";
68-
}
69-
if (isset($params['company'])) {
70-
$sql .= "AND c.name LIKE '".addslashes($params['company'])."%'";
71-
}
72-
if (isset($params['country'])) {
73-
$sql .= "AND c.country LIKE '".addslashes($params['country'])."%'";
74-
}
75-
// And so on... for each parameter, we have a "if" statement
73+
![Sample database schema](doc/images/schema1.png)
74+
75+
Using MagicJoin, you can write this SQL query:
76+
77+
```sql
78+
SELECT users.* FROM MAGICJOIN(users) WHERE groups.name = 'Admins' AND country.name='France';
7679
```
7780

78-
Concatenating SQL queries is **dangerous** (especially if you forget to protect parameters).
79-
You can always use parametrized SQL queries, but you will still have to concatenate the filters.
81+
and it will automatically be transformed into this:
8082

81-
###With Magic-Query
83+
```sql
84+
SELECT users.* FROM users
85+
LEFT JOIN users_groups ON users.user_id = users_groups.user_id
86+
LEFT JOIN groups ON groups.group_id = users_groups.group_id
87+
LEFT JOIN country ON country.country_id = users.country_id
88+
WHERE groups.name = 'Admins' AND country.name='France';
89+
```
8290

83-
```php
84-
// One query with all parameters
85-
$sql = "SELECT * FROM products p JOIN companies c ON p.company_id = c.id WHERE
86-
(p.name LIKE :name OR p.altname LIKE :name)
87-
AND c.name LIKE :company
88-
AND c.country LIKE :country";
91+
And the code is so simple!
8992

90-
$magicQuery = new MagicQuery();
91-
$sql = $magicQuery->build($sql, $params);
92-
```
93+
```php
94+
use Mouf\Database\MagicQuery;
9395

94-
###Other alternatives
96+
$sql = "SELECT users.* FROM MAGICJOIN(users) WHERE groups.name = 'Admins' AND country.name='France'";
9597

96-
To avoid concatenating strings, frameworks and libraries have used different strategies. Using a full ORM (like
97-
Doctrine or Propel) is a good idea, but it makes writing complex queries even more complex. Other frameworks like
98-
Zend are building queries using function calls. These are valid strategies, but you are no more typing SQL queries
99-
directly, and let's face it, it is always useful to use a query directly.
98+
// Get a MagicQuery object.
99+
// $conn is a Doctrine DBAL connection.
100+
$magicQuery = new MagicQuery($conn);
100101

101-
How does it work under the hood?
102-
--------------------------------
102+
$completeSql = $magicQuery->build($sql);
103+
// $completeSql contains the complete SQL request, with all joins.
104+
```
103105

104-
A lot happens to your SQL query. It is actually parsed (thanks to a modified
105-
version of the php-sql-parser library) and then changed into a tree.
106-
The magic happens on the tree where the node containing unused parameters
107-
are simply discarded. When it's done, the tree is changed back to SQL and
108-
"shazam!", your SQL query is purged of useless parameters!
106+
Want to know more? <a class="btn btn-primary" href="doc/magic_join.md">Check out the MagicJoin guide!</a>
109107

110108
Is it a MySQL only tool?
111109
------------------------
112110

113111
No. By default, your SQL is parsed and then rewritten using the MySQL dialect, but you use any kind of dialect
114-
known by Doctrine DBAL. Magic-query optionally uses Doctrine DBAL. You can pass a `Connection` object
112+
known by [Doctrine DBAL](http://docs.doctrine-project.org/projects/doctrine-dbal/en/latest/). Magic-query optionally uses Doctrine DBAL. You can pass a `Connection` object
115113
as the first parameter of the `MagicQuery` constructor. Magic-query will then use the matching dialect.
116114

117115
For instance:
@@ -126,9 +124,24 @@ $conn = \Doctrine\DBAL\DriverManager::getConnection($connectionParams, $config);
126124
$magicQuery = new \Mouf\Database\MagicQuery($conn);
127125
```
128126

127+
What about performances?
128+
------------------------
129+
130+
MagicQuery does a lot to your query. It will parse it, render it internally as a tree of SQL nodes, etc...
131+
This processing is time consuming. So you should definitely consider using a cache system. MagicQuery is compatible
132+
with Doctrine Cache. You simply have to pass a Doctrine Cache instance has the second parameter of the constructor.
133+
134+
```php
135+
use Mouf\Database\MagicQuery;
136+
use Doctrine\Common\Cache\ApcCache();
137+
138+
// $conn is a Doctrine connection
139+
$magicQuery = new MagicQuery($conn, new ApcCache());
140+
```
141+
129142
Any problem?
130143
------------
131144

132-
As we said, a lot happen to your SQL query. In particular, it is parsed using a modified version
145+
With MagicQuery, a lot happens to your SQL query. In particular, it is parsed using a modified version
133146
of the php-sql-parser library. If you face any issues with a complex query, it is likely there is a bug
134147
in the parser. Please open [an issue on Github](https://github.com/thecodingmachine/magic-query/issues) and we'll try to fix it.

‎composer.json

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "mouf/magic-query",
3-
"description": "A very clever library to generate PHP prepared statement with a variable number of parameters... and much more!",
3+
"description": "A very clever library to help you with SQL: generate prepared statements with a variable number of parameters, automatically writes joins... and much more!",
44
"keywords": ["database", "query", "mouf"],
55
"homepage": "http://mouf-php.com/packages/mouf/magic-query",
66
"type": "library",
@@ -17,11 +17,13 @@
1717
"mouf/utils.common.conditioninterface": "~2.0",
1818
"mouf/utils.value.value-interface": "~1.0",
1919
"mouf/utils.common.paginable-interface": "~1.0",
20-
"mouf/utils.common.sortable-interface": "~1.0"
20+
"mouf/utils.common.sortable-interface": "~1.0",
21+
"mouf/schema-analyzer": "~1.0"
2122
},
2223
"require-dev": {
2324
"phpunit/phpunit": "~4.0",
24-
"satooshi/php-coveralls": "dev-master"
25+
"satooshi/php-coveralls": "dev-master",
26+
"doctrine/dbal": "~2.5"
2527
},
2628
"suggest": {
2729
"doctrine/dbal": "To support more databases than just MySQL",
@@ -39,7 +41,8 @@
3941
},
4042
"autoload-dev": {
4143
"psr-4": {
42-
"Mouf\\Database\\": "tests/Mouf/Database/"
44+
"Mouf\\Database\\": "tests/Mouf/Database/",
45+
"SQLParser\\": "tests/SQLParser/"
4346
}
4447
},
4548
"extra": {

‎doc/discard_unused_parameters.md

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
Automatically discard unused parameters
2+
---------------------------------------
3+
4+
###How does it work?
5+
6+
Just write the query with all possible parameters.
7+
8+
```php
9+
use Mouf\Database\MagicQuery;
10+
11+
$sql = "SELECT * FROM users WHERE name LIKE :name AND country LIKE :country";
12+
13+
// Get a MagicQuery object.
14+
$magicQuery = new MagicQuery();
15+
16+
// Let's pass only the "name" parameter
17+
$result = $magicQuery->build($sql, [ "name" => "%John%" ]);
18+
// $result = SELECT * FROM users WHERE name LIKE '%John%'
19+
// Did you notice how the bit about the country simply vanished?
20+
21+
// Let's pass no parameter at all!
22+
$result2 = $magicQuery->build($sql, []);
23+
// $result2 = SELECT * FROM users
24+
// The whole WHERE condition disappeared because it is not needed anymore!
25+
```
26+
27+
###Why should I care?
28+
29+
Because it is **the most efficient way to deal with queries that can have a variable number of parameters**!
30+
Think about a typical datagrid with a bunch of filter (for instance a list of products filtered by name, company, price, ...).
31+
If you have the very common idea to generate the SQL query using no PHP library, your code will look like this:
32+
33+
####Without Magic-query
34+
<div class="alert"><strong>You should not do this!</strong></div>
35+
36+
```php
37+
// People usually write queries like this:
38+
$sql = "SELECT * FROM products p JOIN companies c ON p.company_id = c.id WHERE 1=1 ";
39+
// They keep testing for parameters, and concatenating strings....
40+
if (isset($params['name'])) {
41+
$sql .= "AND (p.name LIKE '".addslashes($params['name'])."%' OR p.altname LIKE '".addslashes($params['name'])."%')";
42+
}
43+
if (isset($params['company'])) {
44+
$sql .= "AND c.name LIKE '".addslashes($params['company'])."%'";
45+
}
46+
if (isset($params['country'])) {
47+
$sql .= "AND c.country LIKE '".addslashes($params['country'])."%'";
48+
}
49+
// And so on... for each parameter, we have a "if" statement
50+
```
51+
52+
Concatenating SQL queries is **dangerous** (especially if you forget to protect parameters).
53+
You can always use parametrized SQL queries, but you will still have to concatenate the filters.
54+
55+
####With Magic-Query
56+
57+
```php
58+
// One query with all parameters
59+
$sql = "SELECT * FROM products p JOIN companies c ON p.company_id = c.id WHERE
60+
(p.name LIKE :name OR p.altname LIKE :name)
61+
AND c.name LIKE :company
62+
AND c.country LIKE :country";
63+
64+
$magicQuery = new MagicQuery();
65+
$sql = $magicQuery->build($sql, $params);
66+
```
67+
68+
###Other alternatives
69+
70+
To avoid concatenating strings, frameworks and libraries have used different strategies. Using a full ORM (like
71+
Doctrine or Propel) is a good idea, but it makes writing complex queries even more complex. Other frameworks like
72+
Zend are building queries using function calls. These are valid strategies, but you are no more typing SQL queries
73+
directly, and let's face it, it is always useful to use a query directly.
74+
75+
How does it work under the hood?
76+
--------------------------------
77+
78+
A lot happens to your SQL query. It is actually parsed (thanks to a modified
79+
version of the php-sql-parser library) and then changed into a tree.
80+
The magic happens on the tree where the node containing unused parameters
81+
are simply discarded. When it's done, the tree is changed back to SQL and
82+
"shazam!", your SQL query is purged of useless parameters!

‎doc/images/schema1.png

17.3 KB
Loading

‎doc/magic_join.md

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
Automatically guess JOINs with MagicJoin!
2+
-----------------------------------------
3+
4+
Fed up of writing joins in SQL? Let MagicQuery do the work for you!
5+
6+
Seriously? Yes! All you have to do is:
7+
8+
- Pass a **Doctrine DBAL connection** to MagicQuery's constructor. MagicQuery will analyze your schema.
9+
- In your SQL query, replace the tables with `magicjoin(start_table)`
10+
- For each column of your query, use the complete name ([table_name].[column_name] instead of [column_name] alone)
11+
12+
Let's assume your database schema is:
13+
14+
![Sample database schema](images/schema1.png)
15+
16+
Using MagicJoin, you can write this SQL query:
17+
18+
```sql
19+
SELECT users.* FROM MAGICJOIN(users) WHERE groups.name = 'Admins' AND country.name='France';
20+
```
21+
22+
and it will automatically be transformed into this:
23+
24+
```sql
25+
SELECT users.* FROM users
26+
LEFT JOIN users_groups ON users.user_id = users_groups.user_id
27+
LEFT JOIN groups ON groups.group_id = users_groups.group_id
28+
LEFT JOIN country ON country.country_id = users.country_id
29+
WHERE groups.name = 'Admins' AND country.name='France';
30+
```
31+
32+
And the code is so simple!
33+
34+
```php
35+
use Mouf\Database\MagicQuery;
36+
37+
$sql = "SELECT users.* FROM MAGICJOIN(users) WHERE groups.name = 'Admins' AND country.name='France'";
38+
39+
// Get a MagicQuery object.
40+
// $conn is a Doctrine DBAL connection.
41+
$magicQuery = new MagicQuery($conn);
42+
43+
$completeSql = $magicQuery->build($sql);
44+
// $completeSql contains the complete SQL request, with all joins.
45+
```
46+
47+
###How does it work?
48+
49+
When you reference a column, you have to give its full name in the form **[table_name].[column_name]**.
50+
Because each column is referenced by its full name, MagicJoin is able to build a list of tables that
51+
need to be joined.
52+
53+
Then, MagicJoin scans your data model. Using **foreign keys**, it discovers the relationship between
54+
tables by itself. Then, it finds the **shortest path** between you "main" table and all the other tables that are
55+
needed.
56+
57+
From this shortest path, it can build the correct JOINs.
58+
59+
Now that you understand the concept, you should understand the power and the limits of MagicJoin.
60+
61+
- MagicJoin **cannot generate queries with recursive relationships** (like a parent/child relationship)
62+
- MagicJoin assumes you are looking for the shortest path between 2 tables
63+
- This is 80% of the case true
64+
- If you are in the remaining 20%, do not use MagicJoin
65+
66+
<div class="alert alert-warning">MagicJoin is meant to be used on the 80% of the cases where writing joins is trivial
67+
and boring. If you have complex joins, do not try to use MagicJoin. Go back to pure SQL instead.</div>
68+
69+
###With great power comes great responsibility
70+
71+
MagicJoin is a powerful feature because it can **guess** what you want to do. Please be wise while using it.
72+
A change in your model could modify the shortest path computed by MagicJoin and therefore the returned SQL query.
73+
74+
If you database model is complex enough and is rapidly changing, generated queries could suddenly change in
75+
your application.
76+
77+
Please consider one of these 2 options:
78+
79+
- write unit tests (you have unit tests, don't you?)
80+
- use MagicJoin in development mode only. While developing, dump the SQL generated by MagicJoin and put it back
81+
in your code.

0 commit comments

Comments
 (0)