Merge pull request #11300 from DANS-KNAW-jp/datatype-for-vtt

content type for files with vtt extension

Author: ofahimIQSS

doc/release-notes/11041-datatype-for-vtt.md

@@ -0,0 +1,43 @@
+### video subtitles (vtt files)
+
+The `IQSS/dataverse` PR sets the content type for new(!) files with extension `vtt` to `text/vtt`
+what is presented as "_Web Video Text Tracks_". The PR also enables full text indexing for these files,
+if [configured](https://guides.dataverse.org/en/latest/installation/config.html#solrfulltextindexing).
+
+The `gdcc/dataverse-previewer` PRs provide a new version of the video previewer. 
+The new previewer version presents `vtt` files as subtitles for videos,
+the naming convention is `<video-basename>.<language-tag>.vtt`.
+The previewer does not rely on the content type.
+A proper content type may hint users to ask permission for the subtitles together with a video.
+
+Existing files with extension `vtt` will keep content type `application/octet-stream` presented as "_Unknown_".
+The following query shows the number of files per extension with an "_Unknown_" content type:
+
+      SELECT substring(m.label from (length(label) - strpos(reverse(m.label), '.') + 2)) AS extension, COUNT(*) as count
+      FROM datafile f LEFT JOIN filemetadata m ON f.id = m.datafile_id
+      WHERE f.contenttype = 'application/octet-stream'
+      GROUP BY extension;
+
+If `vtt` does not appear in the result, you are done. 
+Otherwise, you may want to update the content type for existing files and reindex those datasets.
+
+First figure out which datasets would need [reindexing](https://guides.dataverse.org/en/latest/admin/solr-search-index.html#manual-reindexing):
+
+      select distinct
+        o.protocol, o.authority, o.identifier,
+        v.versionnumber, v.minorversionnumber, v.versionstate
+        from      datafile       f
+        left join filemetadata   m on f.id = m.datafile_id
+        left join datasetversion v on v.id = m.datasetversion_id
+        left join dvobject       o on o.id = v.dataset_id
+        WHERE contenttype = 'application/octet-stream' 
+        AND 'vtt' = substring(m.label from (length(label) - strpos(reverse(m.label), '.') + 2))
+        ;
+
+Then update the content type for the files:
+
+      UPDATE datafile SET contenttype = 'text/vtt' WHERE id IN (
+        SELECT datafile_id FROM filemetadata m
+        WHERE contenttype = 'application/octet-stream' 
+        AND 'vtt' = substring(m.label from (length(label) - strpos(reverse(m.label), '.') + 2))
+      );

src/main/java/propertyFiles/MimeTypeDetectionByFileExtension.properties

@@ -16,6 +16,7 @@ mat=application/matlab-mat
 md=text/markdown
 mp3=audio/mp3
 m4a=audio/mp4
+vtt=text/vtt
 nii=image/nii
 nc=application/netcdf
 ods=application/vnd.oasis.opendocument.spreadsheet

src/main/java/propertyFiles/MimeTypeDisplay.properties

@@ -217,6 +217,7 @@ video/x-m4v=MPEG-4 Video
 video/ogg=OGG Video
 video/quicktime=Quicktime Video
 video/webm=WebM Video
+text/vtt=Web Video Text Tracks
 # Network Data
 text/xml-graphml=GraphML Network Data
 # 3D Data

src/main/java/propertyFiles/MimeTypeFacets.properties

@@ -30,6 +30,7 @@ text/richtext=Text
 text/turtle=Text
 application/xml=Text
 text/xml=Text
+text/vtt=Text
 # Code
 text/x-c=Code
 text/x-c++src=Code