How to check if a 'Storage ' entry exists?

[espruino-discuss]

Posted at 2020-07-29 by @MaBecker

Running some test and figured out that there are at least two possibilities to check if a file/entry exists

s = require('Storage');
file = { name : 'ABC', exist : 0};

// variant I
if (s.list().find(e => (e == file.name))) {
      file.exist = 1;
      console.log(file.name,"exists");
} else {
      file.exist = 1;
      console.log(file.name,"is missing ");
}

// variant II
if (s.read(file.name)) {
      file.exist = 1;
      console.log(file.name,"exists");
} else {
      file.exist = 1;
      console.log(file.name,"is missing ");
}

Are there any pros and cons for those variants?

[gfwilliams]

Posted at 2020-07-29 by @gfwilliams

Or a 3rd is s.list(/^filename$/) which would be more efficient than s.list.

However I'd say s.read(...)!==undefined) is the best. Espruino doesn't actually load the file into memory - it just references it as a string - so it's actually very efficient to just check if it is undefined or not as the file data will never actually even be read.

[espruino-discuss]

Posted at 2020-08-07 by gerardwr

Great, I was just looking for some "file exists" code ;-)

The suggestions work OK for filenames without extension like "ABC". But I want to check for file "news.txt" and all variants report "false/missing".

This is my code

s = require('Storage');
file = { name : 'news.txt', exist : 0};

print(s.list());

// variant I
if (s.list().find(e => (e == file.name))) {
      file.exist = 1;
      console.log("I",file.name,"exists");
} else {
      file.exist = 1;
      console.log("I",file.name,"is missing ");
}
// variant II
if (s.read(file.name)) {
      file.exist = 1;
      console.log("II",file.name,"exists");
} else {
      file.exist = 1;
      console.log("II",file.name,"is missing ");
}

// variant III
print("III",s.list(/^file.name/));

// variant IV
print("IV",s.read(file.name)!==undefined);

// variant V
print("V",s.read(file.name));

Output is
`

[
"news.txt\1",
"weath0.txt\1",
"weath1.txt\1",
"stocks.txt\1",
"messages.txt\1",
"ABC",
"news"
]
I news.txt is missing
II news.txt is missing
III [ ]
IV false
V undefined

`

[espruino-discuss]

Posted at 2020-08-09 by gerardwr

This works for me. It's based on the fact that addr is zero when a file does not exist. When a file exists addr is not zero but points to an address in Storage.

var Storage = require("Storage");

l=Storage.list();
print(l);

function exists(name){
  return Boolean(Storage.open(name,"r").addr);
}

print(exists("news.txt"));
print(exists("nonews.txt"));

The output is:

[
"news.txt\1",
"weath0.txt\1",
"weath1.txt\1",
"stocks.txt\1",
"messages.txt\1"
]
true
false

[espruino-discuss]

Posted at 2020-08-09 by @MaBecker

Thanks for sharing.

This is what I decided to use for my projects:

s = require('Storage');
filename = 'foo';
if (s.read(filename)!==undefined) {
    // file exits
} else {
  // file missing
}

[espruino-discuss]

Posted at 2020-08-09 by gerardwr

OK, thanks, but all of my files have a .txt extension, and those are not recognised by your code :(

s = require('Storage');

print(s.list());

filename = 'news.txt';
if (s.read(filename)!==undefined) {
  print("exists");
} else {
  print("does not exists");
}

Output is:

[
"news.txt\1",
"weath0.txt\1",
"weath1.txt\1",
"stocks.txt\1",
"messages.txt\1"
]
does not exists

[espruino-discuss]

Posted at 2020-08-09 by @fanoush

there are two storage file apis/types of files, the 'more advanced' ones are done via basic ones by appending numbered suffixes Storage.read matches Storage.write, then there is StorageFile created by Storage.open, those are different.

[espruino-discuss]

Posted at 2020-08-09 by @MaBecker

Correct, because searching with 'news.txt' will newer find 'news.txt\1'.

Using StorageFile will add chunk numbering.

[espruino-discuss]

Posted at 2020-08-09 by gerardwr

Fair enough @MaBecker but news.txt\1 is not recognised either.

s = require('Storage');
print(s.list());
filename = 'news.txt\1';
if (s.read(filename)!==undefined) {
  print("exists");
} else {
  print("does not exists");
}

Output is:

[
"news.txt\1",
"weath0.txt\1",
"weath1.txt\1",
"stocks.txt\1",
"messages.txt\1"
]
does not exists

[espruino-discuss]

Posted at 2020-08-09 by gerardwr

When I check for files without an added chunk number (f.i. file ABC.txt) it is recognised perfectly with your code, output is:

[
"news.txt\1",
"weath0.txt\1",
"weath1.txt\1",
"stocks.txt\1",
"messages.txt\1",
"ABC.txt"
]
exists

As @fanoush explained there are 2 methods to store a file, I happened to use the files with the added chunk number.

[espruino-discuss]

Posted at 2020-08-09 by gerardwr

Tested my code once more to verify that both news.txt and news.txt\1 are recognised.
IT DOES NOT :O

var Storage = require("Storage");
l=Storage.list();
print(l);
function exists(name){
  return Boolean(Storage.open(name,"r").addr);
}
print(exists("news.txt"));
print(exists("news.txt\1"));
print(exists("nonews.txt"));
print(exists("ABC.txt"));

Output is:

[
"news.txt\1",
"weath0.txt\1",
"weath1.txt\1",
"stocks.txt\1",
"messages.txt\1",
"ABC.txt"
]
true
false
false
false

The mix of the 2 storage versions does not feel like a blessing anymore :)

[gfwilliams]

Posted at 2020-08-17 by @gfwilliams

The main issue here is the two distinct file types. If you want to check two you could do:

require("Storage").open(name,"r").addr!=0 || require("Storage").read(name)!==undefined

But I guess I wonder what you're trying to do here? If you are writing the code you presumably know which type of file you're trying to create, so you'd know what to check for?

Given the issues around StorageFile I may end up ignoring them in .list at some point and moving to using .listSF or something, so checking for both types of file explicitly (if you have to) is probably the best option.

[espruino-discuss]

Posted at 2020-08-17 by gerardwr

When I started using Espruino a few weeks ago, I was "fresh" re. Espruino. I found the extensive description really confusing. So I just picked one.

Now being a somewhat "less fresh" user I still wonder why there are 2 different file types. I read and reread the documentation, and still have no idea why the differences. It's probably me .....

Just making sure, my posts were not meant as a gripe, but an attempt to add to the discusion that @MaBecker kicked of.

[espruino-discuss]

Posted at 2020-08-18 by @fanoush

I still wonder why there are 2 different file types

The 'more advanced' one is done via basic one. The basic one has less overhead but cannot append to existing file. See also https://www.espruino.com/Reference#StorageFile
So if you just write once or know how many bytes you will write in advance and will write it in chunks you don't need extra features StorageFile gives you. Also with Storage.write you will have continuous block so you can take direct memory pointer of that so Storage.write is more low level.

Given the issues around StorageFile I may end up ignoring them in .list at some point

Maybe there should be Storage.list and StorageFile.list?

[espruino-discuss]

Posted at 2020-08-18 by @MaBecker

Maybe there should be Storage.list and StorageFile.list?

Good point, what about having an option for this?

[gfwilliams]

Posted at 2020-08-18 by @gfwilliams

I'll update the reference, but as @fanoush says, StorageFile is built on top of the standard storage functionality, which makes it harder to separate the two.

Normal Storage functions are what I'd recommend you use for basically everything unless you intend to append textual data (eg a log file). They're much more efficient to write and read.

StorageFile exists because on Bangle.js especially there's a need to log data (GPS readings, heart rate, etc) and using fixed-size files for that is a complete pain.

Maybe there should be Storage.list and StorageFile.list?

The issue is StorageFile is really only something you create with require("Storage").open. So probably it's going to end up being require("Storage").listSF or require("Storage").list({storageFile:true})

[espruino-discuss]

Posted at 2020-08-18 by @fanoush

The issue is StorageFile is really only something you create with require("Storage").open

oh, you can't have static method of StorageFile type without having any instance?

like there is Date.now() vs Date().getHour()

[espruino-discuss]

Posted at 2020-08-18 by gerardwr

Thanks for the extensive explanation, sounds like a perfect addition for the reference. It's clear for me now.

Coming from other languages it feels "natural" to start with an "open" command, so that's the reason why I chose Storagefile in the 1st place.

This says it all!

Normal Storage functions are what I'd recommend you use for basically everything unless you intend to append textual data (eg a log file). They're much more efficient to write and read.

[gfwilliams]

Posted at 2020-08-19 by @gfwilliams

you can't have static method of StorageFile type without having any instance?

Well, you can, but StorageFile shouldn't really be a public object name anyway, since it's only useful via require("Storage").open.

I guess there's a lot to be said for just making Storage public so you can do Storage.open though

[espruino-discuss]

Posted at 2020-08-19 by @fanoush

OK, I am not JS guru, I looked at it from c++/java/c# point of view and was thinking it is related to StorageFile so can be static method there but true that it does not fit there 100% too. Or it may if also open() would move there so Storage and StorageFile would become logically separated APIs - you would use StorageFile.list and StorageFile.open returning new instance not knowing about Storage API at all.

[gfwilliams]

Posted at 2020-08-19 by @gfwilliams

StorageFile.list and StorageFile.open

Yes, that would definitely be the tidiest solution. Then it's just how to make it backwards-compatible enough that it didn't cause problems.