task.album parameter passed to import_task_files event or import stages don't have flexible attributes that I've set

[Neurrone]

Hi,

I'm not sure if this is a bug or if I'm missing something, hence why this is a discussion.

I'm attempting to add fetching of cover art for an Audible plugin. In the candidates function, I have added a cover_url flexible attribute on the AlbumInfo object which stores the url where the image can be downloaded from. This is because in that function while getting a list of possible matches, the API response does contain the cover image url.

I tried adding a new import stage, and accessing the attribute by using task.album.cover_url to no avail. Inspecting the keys on the task.album object, it seems that all flexible attributes that I've set in the candidates function have been stripped, leaving only the built-in attributes.

Is there a way to have the flexible attributes preserved on albums passed into events?

I'm using the fetchart plugin as a reference while implementing this.

[Neurrone]

When I add the following types:

    @property
    def album_types(self):
        return {
            'cover_url': types.String,
            'summary_html': types.String
        }

Beets throws the following error on import even though these 2 attributes are returned in the candidates function, I suspect this is related to the issue that I'm having.

Traceback (most recent call last):                                                                                                   
  File "/usr/bin/beet", line 33, in <module>                                                                                         
    sys.exit(load_entry_point('beets==1.6.0', 'console_scripts', 'beet')())                                                          
  File "/usr/lib/python3.9/site-packages/beets/ui/__init__.py", line 1285, in main                                                   
    _raw_main(args)                                                                                                                  
  File "/usr/lib/python3.9/site-packages/beets/ui/__init__.py", line 1272, in _raw_main                                              
    subcommand.func(lib, suboptions, subargs)                                                                                        
  File "/usr/lib/python3.9/site-packages/beets/ui/commands.py", line 973, in import_func                                             
    import_files(lib, paths, query)                                                                                                  
  File "/usr/lib/python3.9/site-packages/beets/ui/commands.py", line 943, in import_files                                            
    session.run()                                                                                                                    
  File "/usr/lib/python3.9/site-packages/beets/importer.py", line 340, in run                                                        
    pl.run_parallel(QUEUE_SIZE)                                                                                                      
  File "/usr/lib/python3.9/site-packages/beets/util/pipeline.py", line 446, in run_parallel                                          
    raise exc_info[1].with_traceback(exc_info[2])                                                                                    
  File "/usr/lib/python3.9/site-packages/beets/util/pipeline.py", line 311, in run                                                   
    out = self.coro.send(msg)                                                                                                        
  File "/usr/lib/python3.9/site-packages/beets/util/pipeline.py", line 170, in coro                                                  
    task = func(*(args + (task,)))                                                                                                   
  File "/usr/lib/python3.9/site-packages/beets/importer.py", line 1444, in user_query                                                
    apply_choice(session, task)                                                                                                      
  File "/usr/lib/python3.9/site-packages/beets/importer.py", line 1512, in apply_choice                                              
    task.apply_metadata()                                                                                                            
  File "/usr/lib/python3.9/site-packages/beets/importer.py", line 556, in apply_metadata                                             
    autotag.apply_metadata(self.match.info, self.match.mapping)                                                                      
  File "/usr/lib/python3.9/site-packages/beets/autotag/__init__.py", line 192, in apply_metadata                                     
    item[field] = value                                                                                                              
  File "/usr/lib/python3.9/site-packages/beets/library.py", line 617, in __setitem__                                                 
    changed = super()._setitem(key, value)                                                                                           
  File "/usr/lib/python3.9/site-packages/beets/dbcore/db.py", line 403, in _setitem                                                  
    value = self._type(key).normalize(value)                                                                                         
TypeError: normalize() missing 1 required positional argument: 'value'                                                               

[sampsyo]

For this latter thing: you will need to use an object, not a class, for the type. So use types.String(), for example.

[sampsyo]

That's odd; all attributes applied from AlbumInfo objects should persist when the album is added to the database. I'm not exactly sure what could be causing this; digging in to see exactly where the values go after they're applied to the album would be the place to start hunting…

[wisp3rwind]

It appears that we don't actually support applying flexattrs from AlbumInfo to Albums, here's roughly what the relevant code does (unless I'm missing something):

In the user_query pipeline stage, at the very end, apply_choice is called, which calls

• task.apply_metadata: A thin wrapper around autotag.apply_metadata, which sets item fields from AlbumInfo and TrackInfo. In particular, all flexible AlbumInfo fields are assigned to the Items. That's probably questionable. The Album object is not yet created at this point.
• task.add(session.lib): Calls Library.add_album, which stores the tracks to the database and finally creates the Album object. However, this only sets album fields from Album.item_keys. Thus, AlbumInfo flexattrs are never applied to the Album. They're not completely lost, since they were set on the Items in the previous step.

A quick fix could be to set album flexattrs at the end of apply_choice (similar to how set_fields is used there). I think it would be nicer to pass these additional fields to Library.add_album, such that these end up being added in the same db transaction. (In fact, the fields from set_fields could also be part of that transcation.)

(Things that should probably be fixed in bold.)

[sampsyo]

Ah, that would indeed do it. Applying AlbumInfo fields to items instead of albums seems like a straight-up mistake to me.

[wisp3rwind]

Yeah, that's what I thought. But we (ok, rather whoever ends up looking into this...) might want to check the git blame, maybe there's some details on the current implementation.

[wisp3rwind]

One additional observation: This seems to be somewhat of a hack to retain this information (the url) from the metadata source, since this is not really a tag that should be stored permanently in the database, but rather temporary information during import. It would be cleaner to somehow store this in the ImportTask object. However, since this autotagger does not have access to that, this is probably not really possible (currently).