Merge branch 'develop'

* develop:
  We need to make sure we are shifting startsAt after range extraction.
  Hashes may not be ordered at the time of getting ranges or merging.
  Let's make sure we capture some details in case test fails intermittently.
  We actually need AVHashes to operate.
  Let's provide hashes in the result of ReadHashesByTrackId, in order to have a clearer picture regarding length of the fingerprints.
  Let's provide a method that will give us range of hashes from existing hashes.
  Adding the ability to read fingerprints by track ID.
  Update README.md
  Adding a test for consecutively repeating queries.
  Delete _config.yml

Author: AddictedCS

README.md

@@ -56,7 +56,7 @@ The default storage, which comes bundled with _soundfingerprinting_ NuGet packag
     docker run -d -v /persistent-dir:/app/data -p 3399:3399 -p 3340:3340 addictedcs/soundfingerprinting.emy:latest
 
 **Emy** provides a backoffice interface which you can access on port :3340. 
-In order to insert and query **Emy** server please install [SoundFingerprinting.Emy](https://www.nuget.org/packages/SoundFingerprinting.Emy) NuGet package.
+In order to insert and query **Emy** server please install [SoundFingerprinting.Emy][emy-nuget] NuGet package.
 
     Install-Package SoundFingerprinting.Emy
 
@@ -76,6 +76,8 @@ emyModelService.RegisterMatches(queryResult.ResultEntries);
 ```
 Registering matches is now possible with <code>EmyModelService</code>. The results will be displayed in the **Emy** dashboard.
 
+Similarly, [SoundFingerprinting.Emy][emy-nuget] provides `FFmpegAudioService`, which supports a wide variety of formats for both audio and video fingerprinting. More details about `FFmpegAudioService` can be found below.
+
 <img src="https://i.imgur.com/lhqUY74.png" width="800">
 
 If you plan to use **Emy** storage in a commercial project please contact sergiu@emysound.com for details. Enterprise version is ~12.5x faster when number of tracks exceeds ~10K, supports clustering, replication and much more. By using **Emy** you will also support core SoundFingerprinting library and its ongoing development.
@@ -85,6 +87,9 @@ Previous storages are now considered deprecate, as **Emy** is now considered the
 - ***Solr*** non-relational storage [soundfingerprinting.solr](https://github.com/AddictedCS/soundfingerprinting.solr). MIT licensed, useful when the number of tracks does not exceed 5000 tracks [deprecated].
 - ***MSSQL*** [soundfingerprinrint.sql](https://github.com/AddictedCS/soundfingerprinting.sql) [deprecated]. MIT licensed.
 
+### Supported audio formats
+Read [Supported Audio Formats](https://github.com/AddictedCS/soundfingerprinting/wiki/Supported-Audio-Formats) page for details about different audio services and how you can use them in various operating systems.
+
 ### Query result details
 Every `ResultEntry` object will contain the following information:
 - `Track` - matched track from the datastore
@@ -117,11 +122,6 @@ Starting from version 5.1.0 the fingerprints signature has changed to be more re
 ### Version 5.0.0
 Starting from version 5.0.0 _soundfingerprinting_ library supports .NET Standard 2.0. You can run the application not only on Window environment but on any other .NET Standard [compliant](https://docs.microsoft.com/en-us/dotnet/standard/net-standard) runtime.
 
-### List of additional soundfingerprinting integrations
-Default `SoundFingerprintingAudioService` supports only wave file at the input. If you would like to process other formats, consider using below extensions:
-- [SoundFingerprinting.Audio.NAudio](https://www.nuget.org/packages/SoundFingerprinting.Audio.NAudio) - replacement for default `SoundFingerprintingAudioService` audio service. Provides support for *.mp3* audio processing. Runs only on Windows as it uses [NAudio](https://github.com/naudio/NAudio) framework for underlying decoding and resampling.
-- [SoundFingerprinting.Audio.Bass](https://www.nuget.org/packages/SoundFingerprinting.Audio.Bass) - Bass.Net audio library integration, comes as a replacement for default service. Works faster than the default or NAudio, more accurate resampling, supports multiple audio formats (*.wav*, *.ogg*, *.mp3*, *.flac*). [Bass](http://www.un4seen.com) is free for non-comercial use. Recommended for enterprise users.
-- All demo apps are now located in separate git repositories, [duplicates detector](https://github.com/AddictedCS/soundfingerprinting.duplicatesdetector), [sound tools](https://github.com/AddictedCS/soundfingerprinting.soundtools).
 
 ### Algorithm configuration
 Fingerprinting and Querying algorithms can be easily parametrized with corresponding configuration objects passed as parameters on command creation.
@@ -198,4 +198,7 @@ If you want to contribute you are welcome to open issues or discuss on [issues](
 ### License
 The framework is provided under [MIT](https://opensource.org/licenses/MIT) license agreement.
 
-&copy; Soundfingerprinting, 2010-2019, sergiu@emysound.com
+&copy; Soundfingerprinting, 2010-2020, sergiu@emysound.com
+
+
+[emy-nuget]: https://www.nuget.org/packages/SoundFingerprinting.Emy

_config.yml

@@ -11,5 +11,5 @@
 [assembly: AssemblyCulture("")]
 [assembly: ComVisible(false)]
 [assembly: Guid("4cac962e-ebc5-4006-a1e0-7ffb3e2483c2")]
-[assembly: AssemblyVersion("7.4.1.100")]
-[assembly: AssemblyInformationalVersion("7.4.1.100")]
+[assembly: AssemblyVersion("7.4.6.100")]
+[assembly: AssemblyInformationalVersion("7.4.6.100")]

src/SoundFingerprinting.Tests/Properties/AssemblyInfo.cs

@@ -144,6 +144,52 @@ public void CanSerializeAndDeserialize()
             }
         }
 
+        [Test]
+        public void ShouldMergeNonOverlappingSequences()
+        {
+            var dtfi = CultureInfo.GetCultureInfo("en-US").DateTimeFormat;
+            int count = 80;
+            var aStartsAt = DateTime.Parse("01/15/2019 10:00:00", dtfi);
+            var a = new Hashes(GetHashedFingerprints(count), count * 1.48f, aStartsAt);
+            var bStartsAt = DateTime.Parse("01/15/2019 10:02:00", dtfi);
+            var b = new Hashes(GetHashedFingerprints(count), count * 1.48f, bStartsAt);
+            
+            Assert.IsTrue(a.MergeWith(b, out var c, 2 * 1.48f));
+            Assert.AreEqual(count * 2, c.Count);
+            
+            AssertInvariantsForHashes(c, aStartsAt);
+            var rangeA = c.GetRange(aStartsAt, 120);
+            AssertHashesAreEqual(a, rangeA);
+            AssertInvariantsForHashes(rangeA, aStartsAt);
+            var rangeB = c.GetRange(bStartsAt, 120);
+            AssertHashesAreEqual(b, rangeB);
+            AssertInvariantsForHashes(rangeB, bStartsAt);
+        }
+
+        private static void AssertInvariantsForHashes(Hashes hashes, DateTime startsAt)
+        {
+            Assert.AreEqual(startsAt, hashes.RelativeTo);
+            var list = hashes.ToList();
+            Assert.AreEqual(0, list.First().StartsAt);
+            Assert.AreEqual(0, list.First().SequenceNumber);
+            for (int i = 1; i < hashes.Count; ++i)
+            {
+                Assert.IsTrue(list[i].StartsAt >= list[i - 1].StartsAt);
+                Assert.IsTrue(list[i].SequenceNumber >= list[i - 1].SequenceNumber);
+            }
+
+            Assert.AreEqual(hashes.DurationInSeconds, list.Last().StartsAt - list.First().StartsAt + 1.48f, 0.1f);
+        }
+
+        private static void AssertHashesAreEqual(Hashes a, Hashes b)
+        {
+            Assert.AreEqual(a.Count, b.Count);
+            foreach (var tuple in a.Zip(b))
+            {
+                CollectionAssert.AreEqual(tuple.First.HashBins, tuple.Second.HashBins);
+            }
+        }
+
         private static Hashes Deserialize(byte[] buffer)
         {
             using var stream = new MemoryStream(buffer);
@@ -157,11 +203,11 @@ private static byte[] Serialize(Hashes timed)
             return stream.ToArray();
         }
 
-        private static List<HashedFingerprint> GetHashedFingerprints()
+        private static List<HashedFingerprint> GetHashedFingerprints(int count = 100)
         {
             var random = new Random();
             var list = new List<HashedFingerprint>();
-            for (int i = 0; i < 100; ++i)
+            for (int i = 0; i < count; ++i)
             {
                 int[] hashes = new int[25];
                 for (int j = 0; j < 25; ++j)

src/SoundFingerprinting.Tests/Unit/Data/HashesTest.cs

@@ -8,11 +8,39 @@ namespace SoundFingerprinting.Tests.Unit.Query
     using SoundFingerprinting.Builder;
     using SoundFingerprinting.Data;
     using SoundFingerprinting.InMemory;
-    using SoundFingerprinting.Strides;
+    using SoundFingerprinting.Query;
 
     [TestFixture]
     public class QueryCommandTest
     {
+        /**
+         * query repeats consecutively two times in the track, without any pause
+         * t     -----10-----
+         * track 111111011111
+         * query 111111
+         */
+        [Test]
+        public async Task ShouldIdentifyConsecutiveRepeatingSequencesInTrack()
+        {
+            float[] match = TestUtilities.GenerateRandomFloatArray(10 * 5512);
+            float[] twoCopies = new float[match.Length * 2];
+            match.CopyTo(twoCopies, 0);
+            match.CopyTo(twoCopies, match.Length);
+
+            var modelService = new InMemoryModelService();
+            var audioService = new SoundFingerprintingAudioService();
+            await InsertFingerprints(twoCopies, audioService, modelService);
+
+            var result = await GetQueryResult(match, audioService, modelService);
+
+            Assert.AreEqual(2, result.ResultEntries.Count());
+            foreach (var entry in result.ResultEntries)
+            {
+                Assert.AreEqual(0.95, entry.Confidence, 0.05);
+                Assert.AreEqual(10, entry.CoverageWithPermittedGapsLength, 1);
+            }
+        }
+
         /**
          * Long queries, long matches
          * t     -----15-----25-----35-----45-----
@@ -23,30 +51,13 @@ public class QueryCommandTest
         public async Task ShouldIdentifyOnlyOneMatch()
         {
             float[] match = TestUtilities.GenerateRandomFloatArray(10 * 5512);
-
             float[] withJitter = AddJitter(match);
 
             var modelService = new InMemoryModelService();
             var audioService = new SoundFingerprintingAudioService();
 
-            var hashes = await FingerprintCommandBuilder.Instance
-                                    .BuildFingerprintCommand()
-                                    .From(new AudioSamples(withJitter, "Queen", 5512))
-                                    .UsingServices(audioService)
-                                    .Hash();
-
-            modelService.Insert(new TrackInfo("123", "Bohemian Rhapsody", "Queen"), new Hashes(hashes, withJitter.Length / 5512f, DateTime.Now, Enumerable.Empty<string>()));
-
-            var result = await QueryCommandBuilder.Instance
-                                    .BuildQueryCommand()
-                                    .From(new AudioSamples(withJitter, "cnn", 5512))
-                                    .WithQueryConfig(config =>
-                                    {
-                                        config.AllowMultipleMatchesOfTheSameTrackInQuery = true;
-                                        return config;
-                                    })
-                                    .UsingServices(modelService, audioService)
-                                    .Query();
+            await InsertFingerprints(withJitter, audioService, modelService);
+            var result = await GetQueryResult(withJitter, audioService, modelService);
 
             Assert.IsTrue(result.ContainsMatches);
             var entries = result.ResultEntries.OrderBy(entry => entry.QueryMatchStartsAt).ToList();
@@ -71,28 +82,12 @@ public async Task ShouldIdentifyMultipleTracksInSameQuery()
             var modelService = new InMemoryModelService();
             var audioService = new SoundFingerprintingAudioService();
 
-            var hashes = await FingerprintCommandBuilder.Instance
-                .BuildFingerprintCommand()
-                .From(new AudioSamples(match, "Queen", 5512))
-                .UsingServices(audioService)
-                .Hash();
+            await InsertFingerprints(match, audioService, modelService);
 
-            modelService.Insert(new TrackInfo("123", "Bohemian Rhapsody", "Queen"), new Hashes(hashes, match.Length / 5512f, DateTime.Now, Enumerable.Empty<string>()));
+            var result = await GetQueryResult(withJitter, audioService, modelService);
 
-            var result = await QueryCommandBuilder.Instance
-                .BuildQueryCommand()
-                .From(new AudioSamples(withJitter, "cnn", 5512))
-                .WithQueryConfig(config =>
-                {
-                    config.AllowMultipleMatchesOfTheSameTrackInQuery = true;
-                    return config;
-                })
-                .UsingServices(modelService, audioService)
-                .Query();
-            
             Assert.IsTrue(result.ContainsMatches);
             var entries = result.ResultEntries.OrderBy(entry => entry.QueryMatchStartsAt).ToList();
-            
             Assert.AreEqual(2, entries.Count);
             Assert.AreEqual(15d, entries[0].QueryMatchStartsAt, 1f);
             Assert.AreEqual(45d, entries[1].QueryMatchStartsAt, 1f);
@@ -113,25 +108,9 @@ public async Task ShouldIdentifyMultipleRegionsOfTheSameMatch()
             var modelService = new InMemoryModelService();
             var audioService = new SoundFingerprintingAudioService();
 
-            var hashes = await FingerprintCommandBuilder.Instance
-                                    .BuildFingerprintCommand()
-                                    .From(new AudioSamples(withJitter, "Queen", 5512))
-                                    .UsingServices(audioService)
-                                    .Hash();
-
-            modelService.Insert(new TrackInfo("123", "Bohemian Rhapsody", "Queen"), new Hashes(hashes, withJitter.Length / 5512f, DateTime.Now, Enumerable.Empty<string>()));
+            await InsertFingerprints(withJitter, audioService, modelService);
 
-            var result = await QueryCommandBuilder.Instance
-                .BuildQueryCommand()
-                .From(new AudioSamples(match, "cnn", 5512))
-                .WithQueryConfig(config =>
-                {
-                    config.Stride = new IncrementalStaticStride(256);
-                    config.AllowMultipleMatchesOfTheSameTrackInQuery = true;
-                    return config;
-                })
-                .UsingServices(modelService, audioService)
-                .Query();
+            var result = await GetQueryResult(match, audioService, modelService);
 
             Assert.IsTrue(result.ContainsMatches);
             var entries = result.ResultEntries.OrderBy(entry => entry.TrackMatchStartsAt).ToList();
@@ -151,7 +130,7 @@ public async Task ShouldIdentifyMultipleRegionsOfTheSameMatch()
             Assert.AreEqual(45d, entries[1].TrackMatchStartsAt, 1f);
         }
 
-        private float[] AddJitter(float[] match, int beforeSec = 15, int betweenSec = 10, int afterSec = 15)
+        private static float[] AddJitter(float[] match, int beforeSec = 15, int betweenSec = 10, int afterSec = 15)
         {
             float[] before = TestUtilities.GenerateRandomFloatArray(beforeSec * 5512);
             float[] between = TestUtilities.GenerateRandomFloatArray(betweenSec * 5512);
@@ -165,5 +144,30 @@ private float[] AddJitter(float[] match, int beforeSec = 15, int betweenSec = 10
             Buffer.BlockCopy(after, 0, total, sizeof(float) * (before.Length + 2 * match.Length + between.Length), sizeof(float) * after.Length);
             return total;
         }
+
+        private static async Task<QueryResult> GetQueryResult(float[] match, IAudioService audioService, IModelService modelService)
+        {
+            return await QueryCommandBuilder.Instance
+                .BuildQueryCommand()
+                .From(new AudioSamples(match, "cnn", 5512))
+                .WithQueryConfig(config =>
+                {
+                    config.AllowMultipleMatchesOfTheSameTrackInQuery = true;
+                    return config;
+                })
+                .UsingServices(modelService, audioService)
+                .Query();
+        }
+
+        private static async Task InsertFingerprints(float[] audioSamples, IAudioService audioService, IModelService modelService)
+        {
+            var hashes = await FingerprintCommandBuilder.Instance
+                .BuildFingerprintCommand()
+                .From(new AudioSamples(audioSamples, "Queen", 5512))
+                .UsingServices(audioService)
+                .Hash();
+
+            modelService.Insert(new TrackInfo("123", "Bohemian Rhapsody", "Queen"), new Hashes(hashes, audioSamples.Length / 5512f, DateTime.Now, Enumerable.Empty<string>()));
+        }
     }
 }

src/SoundFingerprinting.Tests/Unit/Query/QueryCommandTest.cs

@@ -262,7 +262,7 @@ await QueryCommandBuilder.Instance.BuildRealtimeQueryCommand()
 
             Assert.AreEqual(hashes.Count, list.Select(entry => entry.Count).Sum());
             var merged = Hashes.Aggregate(list, 20d).ToList();
-            Assert.AreEqual(2, merged.Count);
+            Assert.AreEqual(2, merged.Count, $"Hashes:{string.Join(",", merged.Select(_ => $"{_.RelativeTo},{_.DurationInSeconds:0.00}"))}");
             Assert.AreEqual(hashes.Count, merged.Select(entry => entry.Count).Sum());
 
             var aggregated = Hashes.Aggregate(list, double.MaxValue).ToList();

src/SoundFingerprinting.Tests/Unit/Query/RealtimeQueryCommandTest.cs

@@ -1,11 +1,8 @@
 namespace SoundFingerprinting
 {
     using System.Collections.Generic;
-    using System.Linq;
-
     using SoundFingerprinting.DAO;
     using SoundFingerprinting.DAO.Data;
-    using SoundFingerprinting.Data;
 
     public abstract class AdvancedModelService : ModelService, IAdvancedModelService 
     {
@@ -32,16 +29,5 @@ public virtual IEnumerable<SpectralImageData> GetSpectralImagesByTrackReference(
         {
             return spectralImageDao.GetSpectralImagesByTrackReference(trackReference);
         }
-
-        public IList<HashedFingerprint> ReadHashedFingerprintsByTrack(IModelReference trackReference)
-        {
-            return SubFingerprintDao.ReadHashedFingerprintsByTrackReference(trackReference)
-                                    .Select(subFingerprint => new HashedFingerprint(
-                                                subFingerprint.Hashes,
-                                                subFingerprint.SequenceNumber,
-                                                subFingerprint.SequenceAt,
-                                                subFingerprint.OriginalPoint))
-                                    .ToList();
-        }
     }
 }

src/SoundFingerprinting/AdvancedModelService.cs

@@ -0,0 +1,35 @@
+namespace SoundFingerprinting.Data
+{
+    using System;
+    using ProtoBuf;
+
+    [Serializable]
+    [ProtoContract(SkipConstructor = true)]
+    public class AVHashes
+    {
+        public AVHashes(Hashes audioHashes, Hashes videoHashes)
+        {
+            AudioHashes = audioHashes;
+            VideoHashes = videoHashes;
+        }
+
+        [ProtoMember(1)]
+        public Hashes AudioHashes { get; }
+        
+        [ProtoMember(2)]
+        public Hashes VideoHashes { get; }
+        
+        public void Deconstruct(out Hashes audioHashes, out Hashes videoHashes)
+        {
+            audioHashes = AudioHashes;
+            videoHashes = VideoHashes;
+        }
+        
+        public static AVHashes Empty => new AVHashes(Hashes.Empty, Hashes.Empty);
+
+        public override string ToString()
+        {
+            return $"Audio:[{AudioHashes}], Video:[{VideoHashes}]";
+        }
+    }
+}