Dev (#88)

* drop debugging code

* small update for Likelihood, and a test

* fixing up likelihoods

* improve `basemeasure_depth` dispatch

* still some trouble with inferred basemeasure_depth

* clean up `For` dispatch

* simplify _logdensityof

* optimize for Returns{True} case

* rework basemeasure_depth

* aggressive tests passing!!

* drop type-level stuff

* drop help

* license

* affero

* copyright notice

* merge

* Drop Create Commons

* cleanup after merge

* update support computations

* insupport(d::SuperpositionMeasure, x)

* dorp ParamWeighted

* insupport(d::FactoredBase, x)

* export unsafe_logdensityof

* call promote_type instead of promote_rule

* logdensity_def for named tuple product measures

* type annotation for now

* debugging

* drop shows

* speed up mapped arrays

* throw an error for `Union{}` types

* MT tests passing

* updates

* get tests passing

* MIT license for MeasureBase

* bump version

* cleanup

* spacing

* Move ConditionalMeasure to MeasureBase

* add LogarithmicNumbers

* export basemeasure_sequence

* update superpose

* fix logdensity_rel

* remove FIXME (it's fixed!!)

* logdensityof(d::Density, x)

* simplify insupport(::Lebesgue, ::Real)

* clean up

* assume insupport yields Bool

* change logdensity_rel fall-through to warning and return NaN

* update logdensity_rel

* drop old code

* fix warning

* export logdensity_rel

* logdensity_def(μ::Dirac, ν::Dirac, x)

* logdensity_def methods

* drop `static`

* ]add StatsFuns

* Fixing up superposition

* [compat] entries

* trying to speed things up

* bugfixes

* logdensity_rel tests

* logdensity_rel tests

* drop qualifier, and add a test

* more tests

* type constraint in "logdensityof(μ::AbstractMeasure, x)" (was piracy, oops)

* add some docs

* docs

* docs

* typo

* moar speed

* don't export Test

* some more updates

* logdensity_rel for products

* `kleisli` docs

* update instance_type

* instance_type => Core.Typeof

* `powermeasure` bug fix

* fix logdensity_rel bug

* get `commonbase` to take x type into account

* test powers

* commonbase docstring

* deprecate instance_type

* avoid breakage

* switch || terms

* @ifelse macro

* simplify logdensity_rel

* give up on this @ifelse business

* bump version

* working on likelihoods

* update likelihood

* powerweightedmeasure

* powerweighted update

* more powerweighted methods

* bugfix

* dropFactoredBase

* drop FactoredBase

* (::ProductMeasure) | constraint

* update conditional measure

* update Dirac

* move conditional.jl down in the `include`s

* Kleisli => TransitionKernel

* simplify logdensity_def(::PowerMeasure, x)

* rename kleisli.jl to kernel.jl

* update Dirac tests

* update Half

* get tests passing

* update kernel

* Update Project.toml

* no call-site inlining

* restrict single-arg `kernel` to <:ParameterizedMeasure

* export log_likelihood_ratio

* Drop DensityKind(::Likelihood), at least for now

* isfinite(x) instead of x>-Inf

* add `condition` constructor

* EOF newline

* simplify logdensity_def for power measures

* finishing up

* updates

* kernel stuff

* kernel stuff

* update showe methods

* ass a TODO

* use `dot` instead of `sum`

* drop old code

* typo

* formatting

* cleanup

* kernel updates

* uncomment

* bugfix

* drop old code

* pretty printing

* exports, cleanup

* drop old for.jl

* Make DensityKind(::AbstractLikelihood) = IsDensity()

* update Compat version

* Make likelihoods work with Distributions

* _map(f, x::MappedArrays.ReadonlyMappedArray)

* export productmeasure

* AbstractMeasure(::AbstractMeasure)

* fixedrng

* StdNormal

* add SpecialFunctions

* no need to qualify

* update basemeasure

* include stdnormal

* include fixedrng

* update tests

* using SpecialFunctions

* fixing transport_def

* transport_def bugfix

* StdMeasure(::typeof(randn))

* checked_arg  for LebesgueMeasure

* NoTransformOrigin => NoTransportOrigin

* transport interface for pushforwards

* transporting pushforwards

* Use LebesgueMeasure for basemeasure

* updates

* make testvalue fall back on FixedRNG approach

* un-break testvalue

* CI for Juila 1.8

* fixes

* `rand` on a pushforward calls rand on its parent

* LebesgueMeasure => LebesgueBase
CountingMeasure => CountingBase

* tests passing!

* change `invoke` type

* Change `test_interface` to check for 2-arg testvalue

* manually-specifed inverses

* more pushfwd stuff

* A little less wrong

* add mass interface

* pullback

* mass interface

* working on mass interface

* add some `massof` methods

* Maybe <:Number is better for invalidations?

* float instead of Int

* logmassof

* transports for proxies

* drop latent-joint.jl

* drop exports

* Drop `logmassof` for now

* reorganize Lebesgue measure

* IntervalSets

* proxy(::Lebesgue{MeasureBase.RealNumbers}) = LebesgueBase()

* calling a "useproxy" measure calls its proxy

* StdUniform()(s::Interval)

* typo

* (m::AbstractMeasure)(s::Interval)

* bugfix

* comment

* IntervalSets version constraint

* update dynamic_basemeasure_depth

* format

* Calling a measure calls `massof`

* work on massof

* AbstractSuperpositionMeasure

* fix typo

* typo

* format

* docstrings

* remove massof(::PowerWeightedMeasure) method

* make `massof` better

* update testvalue

* formatting

* update _massof

* Update transports for weighted measures

* add chain rules

* invariant mass under transport

* typo

* bugfix

* hasmethod => Tricks.static_hasmethod

* `massof` methods

* roll back tranports for WeightedMeasure

* Improve transport implementation and add product support (#97)

* Improve default transport implementation

Increases type stability.

* Rename NoTransformOrigin to NoTransportOrigin

* Add rrule for _origin_depth

* Fix ambiguities when forwarding NoTransportOrigin and NoTransportOrigin

* Define getdof for product measures

* Generalize test_transport to tuple-valued measures

* Implement transport for tuple-based products

Co-authored-by: Chad Scherrer <[email protected]>

* `@useproxy` delegates `massof`

* drop CI for nightly

* callable densities (#85)

* callable densities

* separate `Density` and `LogDensity`, etc

* bugfix

* move some code around

* format

* updates

* working on densities

* update CI

* bugfix

* formatting

* reorg

* fix typos

* Drop LogDensityMeasure and refactor

* docstring

* inner type constructor with assertion

* type parameters

* 2-arg density_rel and logdensity_rel

* oops

* properties

* typo

* fix ambiguity

* bugfix

* bugfix

* updates

* drop densityof and logdensityof for AbstractDensity

* updates

* update tests

* update

* formatting

* bad calls throw errors

* drop CI for nightly

* Pushfwd-inverses (#98)

* use InverseFunctions.setinverse

* bug fixes

* bugfix

* bugfix

* pushfwd of a pushfwd

* format

* drop old comment

* drop CI for nightly

* working on pushfwd

* bugfix

* inverse(f) => ν.finv

* separate logdensity functions from transport API

* format

* don't unwrap FunctionWithInverse

* drop redundant method

* leave logdensityof alone, instead write unsafe_logdensityof

* more tests

* more work on tests

* tests

* tests

* still messing with tests

* tests passing

* small edits

* formatting

* add some more failing tests

* add atol to isapprox in test

* getdof(μ::PushforwardMeasure) = getdof(transport_origin(μ))

* update atol

* small fix

* drop ((-) ∘ log1p ∘ (-), StdUniform(), StdExponential())

* remove duplicate method

* remove duplicate `include`

* simplify getdof(::PushforwardMeasure)

* Stieltjes measure function (#100)

* smf

* more smf stuff

* transport_to

* Lebesgue smf

* smf for std measures

* format

* transport_to

* oops

* smfinv

* bugfix

* more fixes

* minor refactoring

* formatting

* change x to p

* bugfix

* smfinv(::StdLogistic, p)

* add NoSMF and NoSMFInverse

* roll back some changes

* transport_def methods

* formatting

* another rollback

* make transport_def depend on smf(inv)

* update smf and transports for ::Half

* change `include` order

* test_smf

* more tests

* tests

* add tests

* formatting

* Drop unneeded type parameters

* smfinv => invsmf

* add some inverses

* Base.Fix1 versions

* some more methods

* drop redundant `transport_def`s

* update `pushfwd`

* change name

* add type

* formatting

* fix docstring

* depend on FunctinoChains

* Use fchain

* simplify transport_def for StdLogistic

* simplify transport_def for StdNormal

* drop redundant method

Co-authored-by: Oliver Schulz <[email protected]>

Author: cscherrer

.github/workflows/CI.yml

@@ -25,7 +25,7 @@ jobs:
         version:
           - '1.6'
           - '1.7'
-          - 'nightly'
+          - '1.8'
         os:
           - ubuntu-latest
         arch:

Project.toml

@@ -10,7 +10,9 @@ Compat = "34da2185-b29b-5c13-b0c7-acf172513d20"
 ConstructionBase = "187b0558-2788-49d3-abe0-74a17ed4e7c9"
 DensityInterface = "b429d917-457f-4dbc-8f4c-0cc954292b1d"
 FillArrays = "1a297f60-69ca-5386-bcde-b61e274b549b"
+FunctionChains = "8e6b2b91-af83-483e-ba35-d00930e4cf9b"
 IfElse = "615f187c-cbe4-4ef1-ba3b-2fcf58d6d173"
+IntervalSets = "8197267c-284f-5f27-9208-e0e47529a953"
 InverseFunctions = "3587e190-3f89-42d0-90ee-14403ec27112"
 IrrationalConstants = "92d709cd-6900-40b7-9082-c6be49f344b6"
 LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
@@ -21,6 +23,7 @@ NaNMath = "77ba4419-2d1f-58cd-9bb1-8ffee604a2e3"
 PrettyPrinting = "54e16d92-306c-5ea0-a30b-337be88ac337"
 Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
 Reexport = "189a3867-3050-52da-a836-e630ba90ab69"
+SpecialFunctions = "276daf66-3868-5448-9aa4-cd146d93841b"
 Static = "aedffcd0-7271-4cad-89d0-dc628f76c6d3"
 Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
 Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
@@ -33,15 +36,18 @@ Compat = "3.35, 4"
 ConstructionBase = "1.3"
 DensityInterface = "0.4"
 FillArrays = "0.12, 0.13"
+FunctionChains = "0.1"
 IfElse = "0.1"
-InverseFunctions = "0.1.7"
+IntervalSets = "0.7"
+InverseFunctions = "0.1.8"
 IrrationalConstants = "0.1"
 LogExpFunctions = "0.3"
 LogarithmicNumbers = "1"
 MappedArrays = "0.4"
 NaNMath = "0.3, 1"
 PrettyPrinting = "0.3, 0.4"
 Reexport = "1"
+SpecialFunctions = "2"
 Static = "0.5, 0.6"
 Tricks = "0.1"
 julia = "1.3"

src/MeasureBase.jl

@@ -8,9 +8,16 @@ import Random: gentype
 using Statistics
 using LinearAlgebra
 
+import IntervalSets
+# This seems harder than it should be to get `IntervalSets.:(..)`
+@eval (using IntervalSets: $(Symbol(IntervalSets.:(..))))
+
+using IntervalSets: Interval, width
+
 import DensityInterface: logdensityof
 import DensityInterface: densityof
 import DensityInterface: DensityKind
+using DensityInterface: FuncDensity, LogFuncDensity
 using DensityInterface
 
 using InverseFunctions
@@ -19,13 +26,15 @@ using ChangesOfVariables
 import Base.iterate
 import ConstructionBase
 using ConstructionBase: constructorof
+using IntervalSets
 
 using PrettyPrinting
 const Pretty = PrettyPrinting
 
 using ChainRulesCore
 using FillArrays
 using Static
+using FunctionChains
 
 export ≪
 export gentype
@@ -108,17 +117,18 @@ using Compat
 
 using IrrationalConstants
 
+include("smf.jl")
 include("getdof.jl")
 include("transport.jl")
 include("schema.jl")
 include("splat.jl")
 include("proxies.jl")
 include("kernel.jl")
 include("parameterized.jl")
-include("combinators/half.jl")
 include("domains.jl")
 include("primitive.jl")
 include("utils.jl")
+include("mass-interface.jl")
 # include("absolutecontinuity.jl")
 
 include("primitives/counting.jl")
@@ -144,9 +154,11 @@ include("standard/stdmeasure.jl")
 include("standard/stduniform.jl")
 include("standard/stdexponential.jl")
 include("standard/stdlogistic.jl")
-include("latent-joint.jl")
+include("standard/stdnormal.jl")
+include("combinators/half.jl")
 
 include("rand.jl")
+include("fixedrng.jl")
 
 include("density.jl")
 include("density-core.jl")

src/combinators/half.jl

@@ -26,4 +26,18 @@ logdensity_def(μ::Half, x) = logdensity_def(unhalf(μ), x)
     insupport(unhalf(d), x)
 end
 
-testvalue(::Half) = 1.0
+testvalue(::Type{T}, ::Half) where {T} = one(T)
+
+massof(μ::Half) = massof(unhalf(μ))
+
+function smf(μ::Half, x)
+    2 * smf(μ.parent, max(x, zero(x))) - 1
+end
+
+function invsmf(μ::Half, p)
+    @assert zero(p) ≤ p ≤ one(p)
+    invsmf(μ.parent, (p + 1) / 2)
+end
+
+transport_def(μ::Half, ::StdUniform, p) = invsmf(μ, p)
+transport_def(::StdUniform, μ::Half, x) = smf(μ, x)

src/combinators/likelihood.jl

@@ -202,4 +202,6 @@ more efficient than
 
     logdensityof(ℓ.k(p), ℓ.x) - logdensityof(ℓ.k(q), ℓ.x)
 """
-likelihood_ratio(ℓ::Likelihood, p, q) = exp(ULogarithmic, logdensity_rel(ℓ.k(p), ℓ.k(q), ℓ.x))
+function likelihood_ratio(ℓ::Likelihood, p, q)
+    exp(ULogarithmic, logdensity_rel(ℓ.k(p), ℓ.k(q), ℓ.x))
+end

src/combinators/power.jl

@@ -128,3 +128,5 @@ end
 function checked_arg(μ::PowerMeasure, x::Any)
     throw(ArgumentError("Size of variate doesn't match size of power measure"))
 end
+
+massof(m::PowerMeasure) = massof(m.parent)^prod(m.axes)

src/combinators/product.jl

@@ -16,6 +16,8 @@ function Pretty.tile(μ::AbstractProductMeasure)
     result *= Pretty.literal(")")
 end
 
+massof(m::AbstractProductMeasure) = prod(massof, marginals(m))
+
 export marginals
 
 function Base.:(==)(a::AbstractProductMeasure, b::AbstractProductMeasure)
@@ -159,9 +161,11 @@ marginals(μ::ProductMeasure) = μ.marginals
 
 # TODO: Better `map` support in MappedArrays
 _map(f, args...) = map(f, args...)
-_map(f, x::MappedArrays.ReadonlyMappedArray) = mappedarray(f ∘ x.f, x.data)
+_map(f, x::MappedArrays.ReadonlyMappedArray) = mappedarray(fchain((x.f, f)), x.data)
 
-testvalue(d::AbstractProductMeasure) = _map(testvalue, marginals(d))
+function testvalue(::Type{T}, d::AbstractProductMeasure) where {T}
+    _map(m -> testvalue(T, m), marginals(d))
+end
 
 export ⊗
 
@@ -220,3 +224,16 @@ end
     end
     return true
 end
+
+getdof(d::AbstractProductMeasure) = mapreduce(getdof, +, marginals(d))
+
+function checked_arg(μ::ProductMeasure{<:NTuple{N,Any}}, x::NTuple{N,Any}) where {N}
+    map(checked_arg, marginals(μ), x)
+end
+
+function checked_arg(
+    μ::ProductMeasure{<:NamedTuple{names}},
+    x::NamedTuple{names},
+) where {names}
+    NamedTuple{names}(map(checked_arg, values(marginals(μ)), values(x)))
+end

src/combinators/smart-constructors.jl

@@ -85,7 +85,7 @@ superpose(nt::NamedTuple) = SuperpositionMeasure(nt)
 
 function superpose(μ::T, ν::T) where {T<:AbstractMeasure}
     if μ == ν
-        return weightedmeasure(logtwo, μ)
+        return weightedmeasure(static(float(logtwo)), μ)
     else
         return superpose((μ, ν))
     end

src/combinators/spikemixture.jl

@@ -37,6 +37,6 @@ function Base.rand(rng::AbstractRNG, T::Type, μ::SpikeMixture)
     return (rand(rng, T) < μ.w) * rand(rng, T, μ.m)
 end
 
-testvalue(μ::SpikeMixture) = testvalue(μ.m)
+testvalue(::Type{T}, μ::SpikeMixture) where {T} = zero(T)
 
 insupport(μ::SpikeMixture, x) = dynamic(insupport(μ.m, x)) || iszero(x)

src/combinators/superpose.jl

@@ -4,6 +4,8 @@ using LogExpFunctions
 
 export SuperpositionMeasure
 
+abstract type AbstractSuperpositionMeasure <: AbstractMeasure end
+
 @doc raw"""
     struct SuperpositionMeasure{NT} <: AbstractMeasure
         components :: NT
@@ -24,17 +26,19 @@ Superposition measures satisfy
     \end{aligned}
 ```
 """
-struct SuperpositionMeasure{C} <: AbstractMeasure
+struct SuperpositionMeasure{C} <: AbstractSuperpositionMeasure
     components::C
 end
 
+massof(m::SuperpositionMeasure) = sum(massof, m.components)
+
 function Pretty.tile(d::SuperpositionMeasure)
     result = Pretty.literal("SuperpositionMeasure(")
     result *= Pretty.list_layout([Pretty.tile.(d.components)...])
     result *= Pretty.literal(")")
 end
 
-testvalue(μ::SuperpositionMeasure) = testvalue(first(μ.components))
+testvalue(::Type{T}, μ::SuperpositionMeasure) where {T} = testvalue(T, first(μ.components))
 
 # SuperpositionMeasure(ms :: AbstractMeasure...) = SuperpositionMeasure{X,length(ms)}(ms)