[tpapp]

Add TVSoftPlus and TVCauchy transformations, which have desirable properties: softplus is linear asymptotically at plus infinity, while the Cauchy CDF goes to 0/1 much slower (polynomial) than logistic.

(Whitespace changes in the PR are incidental.)

[devmotion]

Is 0.2.6 needed? AFAICT logπ is available in all 0.2 releases.

Suggested change

	IrrationalConstants = "0.2.6"
	IrrationalConstants = "0.2"

[devmotion]

I'm not sure I understand this sentence - does it refer to TVSoftPlus and TVCauchy? AFAICT e.g. TVScale and TVShift don't have a shorthand with as either.

[devmotion]

The image is the open interval, isn't it (the tests also check 0 < y < 1)?

Suggested change

	Transformation ``x ↦ 1/2 + arctan(x)/π``, mapping the real line to `[0,1]`, which is
	Transformation ``x ↦ 1/2 + arctan(x)/π``, mapping the real line to `(0, 1)`, which is

[devmotion]

T is unused, and transform already calls float. Maybe simplify to

Suggested change

	function _cauchy_cdf(x::T) where {T<:Real}
	_x = float(x)
	if _x > -1
	one(_x) / 2 + atan(_x) / π
	else
	# avoid 1/2 - 1/2 cancellations
	atan(inv(-_x)) / π
	end
	end
	function _cauchy_cdf(x::Real)
	if x > -1
	one(x) / 2 + atan(x) / π
	else
	# avoid 1/2 - 1/2 cancellations
	atan(inv(-x)) / π
	end
	end

[devmotion]

transform converts to float but this line uses x directly. For integers, the generic log1psq(x::Real) = log1p(abs2(x)) overflows: e.g. log1psq(2^40) == 0.

Suggested change

	y, -(log1psq(x) + IrrationalConstants.logπ)
	y, -(log1psq(float(x)) + IrrationalConstants.logπ)

[devmotion]

inverse_and_logjac evaluates the trig kernels twice (sincospi in inverse, and sinpi here). Since sinpi(1 - y) == sinpi(y), maybe the sincospi result could be reused?

Suggested change

	function inverse(::TVCauchy, y::Number)
	@argcheck 0 <= y <= 1 DomainError
	cotpi(p) = ((s, c) = sincospi(p); c / s)
	y > 0.5 ? cotpi(1 - y) : -cotpi(y)
	end
	function inverse_and_logjac(t::TVCauchy, y::Number)
	x = inverse(t, y)
	x, IrrationalConstants.logπ - 2 * log(sinpi(y))
	end
	function inverse(::TVCauchy, y::Number)
	@argcheck 0 <= y <= 1 DomainError
	first(_cauchy_quantile_and_sinpi(y))
	end
	function inverse_and_logjac(::TVCauchy, y::Number)
	@argcheck 0 <= y <= 1 DomainError
	x, s = _cauchy_quantile_and_sinpi(y)
	x, IrrationalConstants.logπ - 2 * log(s)
	end
	# returns the quantile and sinpi(y)
	function _cauchy_quantile_and_sinpi(y)
	flip = y > 0.5
	s, c = sincospi(flip ? 1 - y : y)
	(flip ? c / s : -c / s), s
	end

[devmotion]

Possibly the usual ifelse problem with AD: at x = 0, the discarded branch has derivative inv(-x) → Inf, so reverse mode might end up with 0 * Inf = NaN? Maybe also test with x = 0, to check whether that's problematic?

[devmotion]

randn() never hits the tails. Maybe add a few explicit checks (and TVSoftPlus is exported, so the module prefix shouldn't be needed)?

Suggested change

	test_transformation(TransformVariables.TVSoftPlus(), y -> y > 0)
	t = TVSoftPlus()
	test_transformation(t, y -> y > 0)
	@test transform(t, 1000.0) == 1000.0
	@test inverse(t, 1000.0) == 1000.0
	@test transform(t, -30.0) ≈ exp(-30.0)
	@test inverse(t, exp(-30.0)) ≈ -30.0
	@test @inferred(transform(t, 1f0)) isa Float32

[devmotion]

Similarly, the x ≤ -1 branch is only tested for moderate values. Maybe something like

Suggested change

	test_transformation(TransformVariables.TVCauchy(), y -> 0 < y < 1)
	t = TVCauchy()
	test_transformation(t, y -> 0 < y < 1)
	@test transform(t, -1e300) ≈ inv(π * 1e300)
	@test transform(t, -Inf) == 0
	@test transform(t, Inf) == 1
	@test inverse(t, 0.0) == -Inf
	@test inverse(t, 1.0) == Inf
	@test_throws DomainError inverse(t, 1.5)
	@test @inferred(transform(t, 1f0)) isa Float32
	@test @inferred(inverse(t, 0.3f0)) isa Float32
	@test transform_and_logjac(t, 2^40)[2] ≈ transform_and_logjac(t, 2.0^40)[2]

[devmotion]

With a = 3.1, only the first branch of _cauchy_cdf is compared. Should we also test a negative value? And maybe add TVSoftPlus() and TVCauchy() to the array tests below, so that transform_and_logjac is checked with Reactant as well?