Skip to content

[FIX] OWSql: enforce data download for non PostgreSQL databases#3178

Merged
lanzagar merged 2 commits intobiolab:masterfrom
robertcv:fixes/sql_download
Aug 3, 2018
Merged

[FIX] OWSql: enforce data download for non PostgreSQL databases#3178
lanzagar merged 2 commits intobiolab:masterfrom
robertcv:fixes/sql_download

Conversation

@robertcv
Copy link
Copy Markdown
Collaborator

@robertcv robertcv commented Aug 1, 2018

Issue

Fixes #3043

Description of changes

SqlTable has poor support for non-PostgreSQL databases. This enforces the download of the selected table so that it works as a normal Table.

Includes
  • Code changes
  • Tests
  • Documentation

@robertcv robertcv force-pushed the fixes/sql_download branch from a8dc795 to b39f43b Compare August 1, 2018 11:55
@codecov-io
Copy link
Copy Markdown

codecov-io commented Aug 1, 2018

Codecov Report

Merging #3178 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #3178      +/-   ##
==========================================
+ Coverage   82.54%   82.54%   +<.01%     
==========================================
  Files         337      337              
  Lines       58431    58446      +15     
==========================================
+ Hits        48229    48246      +17     
+ Misses      10202    10200       -2

@lanzagar lanzagar added this to the 3.15 milestone Aug 3, 2018
@lanzagar lanzagar merged commit 2fb5af8 into biolab:master Aug 3, 2018
@Ryujjin
Copy link
Copy Markdown

Ryujjin commented Sep 14, 2018

Thank you for the patch but with the auto download always enabled the data is to large to fit in memory. No data is loaded and thus no widgets can be used to manipulate the information. What is the workaround for working with large databases?

Thanks!

@ajdapretnar
Copy link
Copy Markdown
Contributor

I thought we went with sampling here... But it could be we discarded this option for some reason.

@Ryujjin Did you try using Data Sampler immediately after SQL and sample it yourself?

@Ryujjin
Copy link
Copy Markdown

Ryujjin commented Sep 18, 2018

Thanks for the reply, I need to be able to pick date ranges or time intervals, last I looked the sampler appeared to be a random range but I could be wrong will look at it again.

Thanks!

@lanzagar
Copy link
Copy Markdown
Contributor

If you know what kind of subset of a large data set you want to explore it will probably be best to use an SQL select query to get just that data. This will also be much faster than transferring large amounts of data and sampling on the client.

@robertcv robertcv deleted the fixes/sql_download branch May 31, 2019 11:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants