Skip to content

Commit a3b52ed

Browse files
moufmoufmoufmouf
committed
Working on cache, working on doc
1 parent 9b896f8 commit a3b52ed

File tree

7 files changed

+280
-77
lines changed

7 files changed

+280
-77
lines changed

‎README.md

Lines changed: 75 additions & 63 deletions
Original file line numberDiff line numberDiff line change
@@ -7,13 +7,33 @@
77
What is Magic-query?
88
====================
99

10-
Magic-query is a PHP library that helps you work with complex queries that require
11-
a variable number of parameters.
10+
Magic-query is a PHP library that helps you work with complex SQL queries.
1211

13-
How does it work?
14-
-----------------
12+
It comes with 2 great features:
1513

16-
Easy! You write the query with all possible parameters.
14+
- [it helps you work with that require a variable number of parameters.](#parameters)
15+
- [**MagicJoin**: it writes JOINs for you!](#joins)
16+
17+
Installation
18+
------------
19+
20+
Simply use the composer package:
21+
22+
```json
23+
{
24+
"require": {
25+
"mouf/magic-query": "~1.0"
26+
},
27+
"minimum-stability": "dev",
28+
"prefer-stable": true
29+
}
30+
```
31+
32+
<a name="parameters"></a>
33+
Automatically discard unused parameters
34+
---------------------------------------
35+
36+
Just write the query with all possible parameters.
1737

1838
```php
1939
use Mouf\Database\MagicQuery;
@@ -34,78 +54,55 @@ $result2 = $magicQuery->build($sql, []);
3454
// The whole WHERE condition disappeared because it is not needed anymore!
3555
```
3656

37-
Installation
38-
------------
57+
Curious to know how this work? <a class="btn btn-primary" href="doc/discard_unused_parameters.md">Check out the complete guide!</a>
3958

40-
Simply use the composer package:
59+
<a name="joins"></a>
60+
Automatically guess JOINs with MagicJoin!
61+
-----------------------------------------
4162

42-
```json
43-
{
44-
"require": {
45-
"mouf/magic-query": "~1.0"
46-
},
47-
"minimum-stability": "dev",
48-
"prefer-stable": true
49-
}
50-
```
63+
Fed up of writing joins in SQL? Let MagicQuery do the work for you!
5164

52-
Why should I care?
53-
------------------
65+
Seriously? Yes! All you have to do is:
5466

55-
Because it is **the most efficient way to deal with queries that can have a variable number of parameters**!
56-
Think about a typical datagrid with a bunch of filter (for instance a list of products filtered by name, company, price, ...).
57-
If you have the very common idea to generate the SQL query using no PHP library, your code will look like this:
67+
- Pass a **Doctrine DBAL connection** to MagicQuery's constructor. MagicQuery will analyze your schema.
68+
- In your SQL query, replace the tables with `magicjoin(start_table)`
5869

59-
###Without Magic-query
60-
<div class="alert"><strong>You should not do this!</strong></div>
70+
Let's assume your database schema is:
6171

62-
```php
63-
// People usually write queries like this:
64-
$sql = "SELECT * FROM products p JOIN companies c ON p.company_id = c.id WHERE 1=1 ";
65-
// They keep testing for parameters, and concatenating strings....
66-
if (isset($params['name'])) {
67-
$sql .= "AND (p.name LIKE '".addslashes($params['name'])."%' OR p.altname LIKE '".addslashes($params['name'])."%')";
68-
}
69-
if (isset($params['company'])) {
70-
$sql .= "AND c.name LIKE '".addslashes($params['company'])."%'";
71-
}
72-
if (isset($params['country'])) {
73-
$sql .= "AND c.country LIKE '".addslashes($params['country'])."%'";
74-
}
75-
// And so on... for each parameter, we have a "if" statement
72+
![Sample database schema](doc/images/schema1.png)
73+
74+
Using MagicJoin, you can write this SQL query:
75+
76+
```sql
77+
SELECT users.* FROM MAGICJOIN(users) WHERE groups.name = 'Admins' AND country.name='France';
7678
```
7779

78-
Concatenating SQL queries is **dangerous** (especially if you forget to protect parameters).
79-
You can always use parametrized SQL queries, but you will still have to concatenate the filters.
80+
and it will automatically be transformed into this:
8081

81-
###With Magic-Query
82+
```sql
83+
SELECT users.* FROM users
84+
LEFT JOIN users_groups ON users.user_id = users_groups.user_id
85+
LEFT JOIN groups ON groups.group_id = users_groups.group_id
86+
LEFT JOIN country ON country.country_id = users.country_id
87+
WHERE groups.name = 'Admins' AND country.name='France';
88+
```
8289

83-
```php
84-
// One query with all parameters
85-
$sql = "SELECT * FROM products p JOIN companies c ON p.company_id = c.id WHERE
86-
(p.name LIKE :name OR p.altname LIKE :name)
87-
AND c.name LIKE :company
88-
AND c.country LIKE :country";
90+
And the code is so simple!
8991

90-
$magicQuery = new MagicQuery();
91-
$sql = $magicQuery->build($sql, $params);
92-
```
92+
```php
93+
use Mouf\Database\MagicQuery;
9394

94-
###Other alternatives
95+
$sql = "SELECT users.* FROM MAGICJOIN(users) WHERE groups.name = 'Admins' AND country.name='France'";
9596

96-
To avoid concatenating strings, frameworks and libraries have used different strategies. Using a full ORM (like
97-
Doctrine or Propel) is a good idea, but it makes writing complex queries even more complex. Other frameworks like
98-
Zend are building queries using function calls. These are valid strategies, but you are no more typing SQL queries
99-
directly, and let's face it, it is always useful to use a query directly.
97+
// Get a MagicQuery object.
98+
// $conn is a Doctrine DBAL connection.
99+
$magicQuery = new MagicQuery($conn);
100100

101-
How does it work under the hood?
102-
--------------------------------
101+
$completeSql = $magicQuery->build($sql);
102+
// $completeSql contains the complete SQL request, with all joins.
103+
```
103104

104-
A lot happens to your SQL query. It is actually parsed (thanks to a modified
105-
version of the php-sql-parser library) and then changed into a tree.
106-
The magic happens on the tree where the node containing unused parameters
107-
are simply discarded. When it's done, the tree is changed back to SQL and
108-
"shazam!", your SQL query is purged of useless parameters!
105+
Want to know more? <a class="btn btn-primary" href="doc/magic_join.md">Check out the MagicJoin guide!</a>
109106

110107
Is it a MySQL only tool?
111108
------------------------
@@ -126,9 +123,24 @@ $conn = \Doctrine\DBAL\DriverManager::getConnection($connectionParams, $config);
126123
$magicQuery = new \Mouf\Database\MagicQuery($conn);
127124
```
128125

126+
What about performances?
127+
------------------------
128+
129+
MagicQuery does a lot to your query. It will parse it, render it internally as a tree of SQL nodes, etc...
130+
This processing is time consuming. So you should definitely consider using a cache system. MagicQuery is compatible
131+
with Doctrine Cache. You simply have to pass a Doctrine Cache instance has the second parameter of the constructor.
132+
133+
```php
134+
use Mouf\Database\MagicQuery;
135+
use Doctrine\Common\Cache\ApcCache();
136+
137+
// $conn is a Doctrine connection
138+
$magicQuery = new MagicQuery($conn, new ApcCache());
139+
```
140+
129141
Any problem?
130142
------------
131143

132-
As we said, a lot happen to your SQL query. In particular, it is parsed using a modified version
144+
With MagicQuery, a lot happens to your SQL query. In particular, it is parsed using a modified version
133145
of the php-sql-parser library. If you face any issues with a complex query, it is likely there is a bug
134146
in the parser. Please open [an issue on Github](https://github.com/thecodingmachine/magic-query/issues) and we'll try to fix it.

‎doc/discard_unused_parameters.md

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
Automatically discard unused parameters
2+
---------------------------------------
3+
4+
###How does it work?
5+
6+
Just write the query with all possible parameters.
7+
8+
```php
9+
use Mouf\Database\MagicQuery;
10+
11+
$sql = "SELECT * FROM users WHERE name LIKE :name AND country LIKE :country";
12+
13+
// Get a MagicQuery object.
14+
$magicQuery = new MagicQuery();
15+
16+
// Let's pass only the "name" parameter
17+
$result = $magicQuery->build($sql, [ "name" => "%John%" ]);
18+
// $result = SELECT * FROM users WHERE name LIKE '%John%'
19+
// Did you notice how the bit about the country simply vanished?
20+
21+
// Let's pass no parameter at all!
22+
$result2 = $magicQuery->build($sql, []);
23+
// $result2 = SELECT * FROM users
24+
// The whole WHERE condition disappeared because it is not needed anymore!
25+
```
26+
27+
###Why should I care?
28+
29+
Because it is **the most efficient way to deal with queries that can have a variable number of parameters**!
30+
Think about a typical datagrid with a bunch of filter (for instance a list of products filtered by name, company, price, ...).
31+
If you have the very common idea to generate the SQL query using no PHP library, your code will look like this:
32+
33+
####Without Magic-query
34+
<div class="alert"><strong>You should not do this!</strong></div>
35+
36+
```php
37+
// People usually write queries like this:
38+
$sql = "SELECT * FROM products p JOIN companies c ON p.company_id = c.id WHERE 1=1 ";
39+
// They keep testing for parameters, and concatenating strings....
40+
if (isset($params['name'])) {
41+
$sql .= "AND (p.name LIKE '".addslashes($params['name'])."%' OR p.altname LIKE '".addslashes($params['name'])."%')";
42+
}
43+
if (isset($params['company'])) {
44+
$sql .= "AND c.name LIKE '".addslashes($params['company'])."%'";
45+
}
46+
if (isset($params['country'])) {
47+
$sql .= "AND c.country LIKE '".addslashes($params['country'])."%'";
48+
}
49+
// And so on... for each parameter, we have a "if" statement
50+
```
51+
52+
Concatenating SQL queries is **dangerous** (especially if you forget to protect parameters).
53+
You can always use parametrized SQL queries, but you will still have to concatenate the filters.
54+
55+
####With Magic-Query
56+
57+
```php
58+
// One query with all parameters
59+
$sql = "SELECT * FROM products p JOIN companies c ON p.company_id = c.id WHERE
60+
(p.name LIKE :name OR p.altname LIKE :name)
61+
AND c.name LIKE :company
62+
AND c.country LIKE :country";
63+
64+
$magicQuery = new MagicQuery();
65+
$sql = $magicQuery->build($sql, $params);
66+
```
67+
68+
###Other alternatives
69+
70+
To avoid concatenating strings, frameworks and libraries have used different strategies. Using a full ORM (like
71+
Doctrine or Propel) is a good idea, but it makes writing complex queries even more complex. Other frameworks like
72+
Zend are building queries using function calls. These are valid strategies, but you are no more typing SQL queries
73+
directly, and let's face it, it is always useful to use a query directly.
74+
75+
How does it work under the hood?
76+
--------------------------------
77+
78+
A lot happens to your SQL query. It is actually parsed (thanks to a modified
79+
version of the php-sql-parser library) and then changed into a tree.
80+
The magic happens on the tree where the node containing unused parameters
81+
are simply discarded. When it's done, the tree is changed back to SQL and
82+
"shazam!", your SQL query is purged of useless parameters!

‎doc/images/schema1.png

17.3 KB
Loading

‎doc/magic_join.md

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
Automatically guess JOINs with MagicJoin!
2+
-----------------------------------------
3+
4+
Fed up of writing joins in SQL? Let MagicQuery do the work for you!
5+
6+
Seriously? Yes! All you have to do is:
7+
8+
- Pass a **Doctrine DBAL connection** to MagicQuery's constructor. MagicQuery will analyze your schema.
9+
- In your SQL query, replace the tables with `magicjoin(start_table)`
10+
11+
Let's assume your database schema is:
12+
13+
![Sample database schema](doc/images/schema1.png)
14+
15+
Using MagicJoin, you can write this SQL query:
16+
17+
```sql
18+
SELECT users.* FROM MAGICJOIN(users) WHERE groups.name = 'Admins' AND country.name='France';
19+
```
20+
21+
and it will automatically be transformed into this:
22+
23+
```sql
24+
SELECT users.* FROM users
25+
LEFT JOIN users_groups ON users.user_id = users_groups.user_id
26+
LEFT JOIN groups ON groups.group_id = users_groups.group_id
27+
LEFT JOIN country ON country.country_id = users.country_id
28+
WHERE groups.name = 'Admins' AND country.name='France';
29+
```
30+
31+
And the code is so simple!
32+
33+
```php
34+
use Mouf\Database\MagicQuery;
35+
36+
$sql = "SELECT users.* FROM MAGICJOIN(users) WHERE groups.name = 'Admins' AND country.name='France'";
37+
38+
// Get a MagicQuery object.
39+
// $conn is a Doctrine DBAL connection.
40+
$magicQuery = new MagicQuery($conn);
41+
42+
$completeSql = $magicQuery->build($sql);
43+
// $completeSql contains the complete SQL request, with all joins.
44+
```
45+
46+
###How does it work?
47+
48+
When you reference a column, you have to give its full name in the form **[table_name].[column_name]**.
49+
Because each column is referenced by its full name, MagicJoin is able to build a list of tables that
50+
need to be joined.
51+
52+
Then, MagicJoin scans your data model. Using **foreign keys**, it discovers the relationship between
53+
tables by itself. Then, it finds the **shortest path** between you "main" table and all the other tables that are
54+
needed.
55+
56+
From this shortest path, it can build the correct JOINs.
57+
58+
Now that you understand the concept, you should understand the power and the limits of MagicJoin.
59+
60+
- MagicJoin **cannot generate queries with recursive relationships** (like a parent/child relationship)
61+
- MagicJoin assumes you are looking for the shortest path between 2 tables
62+
- This is 80% of the case true
63+
- If you are in the remaining 20%, do not use MagicJoin
64+
65+
<div class="alert alert-warning">MagicJoin is meant to be used on the 80% of the cases where writing joins is trivial
66+
and boring. If you have complex joins, do not try to use MagicJoin. Go back to pure SQL instead.</div>
67+

‎src/Mouf/Database/MagicQuery.php

Lines changed: 16 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33
namespace Mouf\Database;
44

5+
use Doctrine\Common\Cache\VoidCache;
56
use Mouf\Database\SchemaAnalyzer\SchemaAnalyzer;
67
use SQLParser\Node\ColRef;
78
use SQLParser\Node\Equal;
@@ -34,7 +35,11 @@ class MagicQuery
3435
public function __construct($connection = null, $cache = null, SchemaAnalyzer $schemaAnalyzer = null)
3536
{
3637
$this->connection = $connection;
37-
$this->cache = $cache;
38+
if ($cache) {
39+
$this->cache = $cache;
40+
} else {
41+
$this->cache = new VoidCache();
42+
}
3843
if ($schemaAnalyzer) {
3944
$this->schemaAnalyzer = $schemaAnalyzer;
4045
}
@@ -67,13 +72,9 @@ public function build($sql, array $parameters = array())
6772
* @throws MagicQueryParserException
6873
*/
6974
public function parse($sql) {
70-
$select = false;
71-
72-
if ($this->cache !== null) {
73-
// We choose md4 because it is fast.
74-
$cacheKey = "request_".hash("md4", $sql);
75-
$select = $this->cache->fetch($cacheKey);
76-
}
75+
// We choose md4 because it is fast.
76+
$cacheKey = "request_".hash("md4", $sql);
77+
$select = $this->cache->fetch($cacheKey);
7778

7879
if ($select === false) {
7980
$parser = new SQLParser();
@@ -87,10 +88,8 @@ public function parse($sql) {
8788

8889
$this->magicJoin($select);
8990

90-
if ($this->cache !== null) {
91-
// Let's store the tree
92-
$this->cache->save($cacheKey, $select);
93-
}
91+
// Let's store the tree
92+
$this->cache->save($cacheKey, $select);
9493
}
9594
return $select;
9695
}
@@ -213,8 +212,12 @@ private function getSchemaAnalyzer() {
213212
throw new MagicQueryMissingConnectionException('In order to use MagicJoin, you need to configure a DBAL connection.');
214213
}
215214

216-
$this->schemaAnalyzer = new SchemaAnalyzer($this->connection->getSchemaManager()->createSchema());
215+
$this->schemaAnalyzer = new SchemaAnalyzer($this->connection->getSchemaManager(), $this->cache, $this->getConnectionUniqueId());
217216
}
218217
return $this->schemaAnalyzer;
219218
}
219+
220+
private function getConnectionUniqueId() {
221+
return hash('md4', $this->connection->getHost()."-".$this->connection->getPort()."-".$this->connection->getDatabase()."-".$this->connection->getDriver()->getName());
222+
}
220223
}

0 commit comments

Comments
 (0)