|
310 | 310 | "cell_type": "markdown", |
311 | 311 | "source": [ |
312 | 312 | "Some exciting news is, the next upcoming version of the data algebra\n", |
313 | | - "(version `0.8.3`) incorporate this simulation into its SQLite\n", |
| 313 | + "(version `0.8.3` and above) incorporate this simulation into its SQLite\n", |
314 | 314 | "adapter. It performs and operator tree to dag re-write and can execute\n", |
315 | 315 | "the original operations directly in the database." |
316 | 316 | ], |
|
340 | 340 | { |
341 | 341 | "cell_type": "markdown", |
342 | 342 | "source": [ |
343 | | - "And that is how to simulate a full join using concatenate and left-join." |
| 343 | + "The generated query isn't too illegible. And due to power of SQL-with can in fact be used on\n", |
| 344 | + "non-table sources." |
344 | 345 | ], |
345 | 346 | "metadata": { |
346 | 347 | "collapsed": false, |
|
352 | 353 | { |
353 | 354 | "cell_type": "code", |
354 | 355 | "execution_count": 10, |
| 356 | + "outputs": [ |
| 357 | + { |
| 358 | + "name": "stdout", |
| 359 | + "output_type": "stream", |
| 360 | + "text": [ |
| 361 | + "-- data_algebra SQL https://github.com/WinVector/data_algebra\n", |
| 362 | + "-- dialect: SQLiteModel\n", |
| 363 | + "-- string quote: '\n", |
| 364 | + "-- identifier quote: \"\n", |
| 365 | + "WITH\n", |
| 366 | + " \"table_reference_0\" AS (\n", |
| 367 | + " SELECT\n", |
| 368 | + " \"g\"\n", |
| 369 | + " FROM\n", |
| 370 | + " \"d1\"\n", |
| 371 | + " ) ,\n", |
| 372 | + " \"project_1\" AS (\n", |
| 373 | + " SELECT -- .project({ }, group_by=['g'])\n", |
| 374 | + " \"g\"\n", |
| 375 | + " FROM\n", |
| 376 | + " \"table_reference_0\"\n", |
| 377 | + " GROUP BY\n", |
| 378 | + " \"g\"\n", |
| 379 | + " ) ,\n", |
| 380 | + " \"table_reference_2\" AS (\n", |
| 381 | + " SELECT\n", |
| 382 | + " \"g\"\n", |
| 383 | + " FROM\n", |
| 384 | + " \"d2\"\n", |
| 385 | + " ) ,\n", |
| 386 | + " \"project_3\" AS (\n", |
| 387 | + " SELECT -- .project({ }, group_by=['g'])\n", |
| 388 | + " \"g\"\n", |
| 389 | + " FROM\n", |
| 390 | + " \"table_reference_2\"\n", |
| 391 | + " GROUP BY\n", |
| 392 | + " \"g\"\n", |
| 393 | + " ) ,\n", |
| 394 | + " \"concat_rows_4\" AS (\n", |
| 395 | + " SELECT -- _0..concat_rows(b= _1, id_column=None, a_name='a', b_name='b')\n", |
| 396 | + " \"g\"\n", |
| 397 | + " FROM\n", |
| 398 | + " (\n", |
| 399 | + " SELECT\n", |
| 400 | + " *\n", |
| 401 | + " FROM\n", |
| 402 | + " \"project_1\"\n", |
| 403 | + " UNION ALL\n", |
| 404 | + " SELECT\n", |
| 405 | + " *\n", |
| 406 | + " FROM\n", |
| 407 | + " \"project_3\"\n", |
| 408 | + " ) \"concat_rows_4\"\n", |
| 409 | + " ) ,\n", |
| 410 | + " \"project_5\" AS (\n", |
| 411 | + " SELECT -- .project({ }, group_by=['g'])\n", |
| 412 | + " \"g\"\n", |
| 413 | + " FROM\n", |
| 414 | + " \"concat_rows_4\"\n", |
| 415 | + " GROUP BY\n", |
| 416 | + " \"g\"\n", |
| 417 | + " ) ,\n", |
| 418 | + " \"natural_join_6\" AS (\n", |
| 419 | + " SELECT -- _0..natural_join(b= _1, by=['g'], jointype='LEFT')\n", |
| 420 | + " COALESCE(\"project_5\".\"g\", \"d1\".\"g\") AS \"g\" ,\n", |
| 421 | + " \"v1\" ,\n", |
| 422 | + " \"v2\"\n", |
| 423 | + " FROM\n", |
| 424 | + " (\n", |
| 425 | + " \"project_5\"\n", |
| 426 | + " LEFT JOIN\n", |
| 427 | + " \"d1\"\n", |
| 428 | + " ON\n", |
| 429 | + " \"project_5\".\"g\" = \"d1\".\"g\"\n", |
| 430 | + " )\n", |
| 431 | + " )\n", |
| 432 | + "SELECT -- _0..natural_join(b= _1, by=['g'], jointype='LEFT')\n", |
| 433 | + " COALESCE(\"natural_join_6\".\"g\", \"d2\".\"g\") AS \"g\" ,\n", |
| 434 | + " COALESCE(\"natural_join_6\".\"v1\", \"d2\".\"v1\") AS \"v1\" ,\n", |
| 435 | + " COALESCE(\"natural_join_6\".\"v2\", \"d2\".\"v2\") AS \"v2\"\n", |
| 436 | + "FROM\n", |
| 437 | + "(\n", |
| 438 | + " \"natural_join_6\"\n", |
| 439 | + "LEFT JOIN\n", |
| 440 | + " \"d2\"\n", |
| 441 | + "ON\n", |
| 442 | + " \"natural_join_6\".\"g\" = \"d2\".\"g\"\n", |
| 443 | + ")\n", |
| 444 | + "\n" |
| 445 | + ] |
| 446 | + } |
| 447 | + ], |
| 448 | + "source": [ |
| 449 | + "print(sqlite_handle.to_sql(ops))" |
| 450 | + ], |
| 451 | + "metadata": { |
| 452 | + "collapsed": false, |
| 453 | + "pycharm": { |
| 454 | + "name": "#%%\n" |
| 455 | + } |
| 456 | + } |
| 457 | + }, |
| 458 | + { |
| 459 | + "cell_type": "markdown", |
| 460 | + "source": [ |
| 461 | + "And that is how to simulate a full join using concatenate and left-join." |
| 462 | + ], |
| 463 | + "metadata": { |
| 464 | + "collapsed": false, |
| 465 | + "pycharm": { |
| 466 | + "name": "#%% md\n" |
| 467 | + } |
| 468 | + } |
| 469 | + }, |
| 470 | + { |
| 471 | + "cell_type": "code", |
| 472 | + "execution_count": 11, |
355 | 473 | "outputs": [], |
356 | 474 | "source": [ |
357 | 475 | "# clean up\n", |
|
0 commit comments