Hostname: page-component-cb9f654ff-5kfdg Total loading time: 0 Render date: 2025-08-05T10:23:05.750Z Has data issue: false hasContentIssue false

Inverse clustering of Gibbs partitions via independent fragmentation and dual dependent coagulation operators

Published online by Cambridge University Press:  30 July 2025

Man-Wai Ho*
Affiliation:
The Hong Kong University of Science and Technology
Lancelot F. James*
Affiliation:
The Hong Kong University of Science and Technology
John W. Lau*
Affiliation:
The University of Western Australia
*
*Postal address: Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong SAR.
*Postal address: Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong SAR.
***Postal address: UWA Centre for Applied Statistics, The University of Western Australia (M019), 35 Stirling Highway, CRAWLEY WA 6009, Australia.

Abstract

Exchangeable partitions of the integers and their corresponding mass partitions on $\mathcal{P}_{\infty} = \{\mathbf{s} = (s_{1},s_{2},\ldots)\colon s_{1} \geq s_{2} \geq \cdots \geq 0$ and $\sum_{k=1}^{\infty}s_{k} = 1\}$ play a vital role in combinatorial stochastic processes and their applications. In this work, we continue our focus on the class of Gibbs partitions of the integers and the corresponding stable Poisson–Kingman-distributed mass partitions generated by the normalized jumps of a stable subordinator with an index $\alpha \in (0,1)$, subject to further mixing. This remarkable class of infinitely exchangeable random partitions is characterized by probabilities that have Gibbs (product) form. These partitions have practical applications in combinatorial stochastic processes, random tree/graph growth models, and Bayesian statistics. The most notable class consists of random partitions generated from the two-parameter Poisson–Dirichlet distribution $\mathrm{PD}(\alpha,\theta)$. While the utility of Gibbs partitions is recognized, there is limited understanding of the broader class. Here, as a continuation of our work, we address this gap by extending the dual coagulation/fragmentation results of Pitman (1999), developed for the Poisson–Dirichlet family, to all Gibbs models and their corresponding Poisson–Kingman mass partitions, creating nested families of Gibbs partitions and mass partitions. We focus primarily on fragmentation operations, identifying which classes correspond to these operations and providing significant calculations for the resulting Gibbs partitions. Furthermore, for completion, we provide definitive results for dual coagulation operations using dependent processes. We demonstrate the applicability of our results by establishing new findings for Brownian excursion partitions, Mittag-Leffler, and size-biased generalized gamma models.

Information

Type
Original Article
Copyright
© The Author(s), 2025. Published by Cambridge University Press on behalf of Applied Probability Trust

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Bacallado, S., Battiston, M., Favaro, S. and Trippa, L. (2017). Sufficientness postulates for Gibbs-type priors and hierarchical generalizations. Statist. Sci. 32, 487500.10.1214/17-STS619CrossRefGoogle Scholar
Basdevant, A. L. (2006). Ruelleś probability cascades seen as a fragmentation process. Markov Process. Relat. Fields 12, 447474.Google Scholar
Bertoin, J. (2006). Random Fragmentation and Coagulation Processes. Cambridge University Press.10.1017/CBO9780511617768CrossRefGoogle Scholar
Bertoin, J. and Le Gall, J.-F. (2000). The Bolthausen–Sznitman coalescent and the genealogy of continuous-state branching processes. Prob. Theory Relat. Fields 117, 249266.10.1007/s004400050006CrossRefGoogle Scholar
Bertoin, J. and Pitman, J. (2000). Two coalescents derived from the ranges of stable subordinators. Electron. J. Prob. 5, 117.10.1214/EJP.v5-63CrossRefGoogle Scholar
Bolthausen, E. and Sznitman, A. S. (1998). On Ruelleś probability cascades and an abstract cavity method. Commun. Math. Phys. 197, 247276.10.1007/s002200050450CrossRefGoogle Scholar
Caron, F. and Fox, E. B. (2017). Sparse graphs using exchangeable random measures. J. R. Statist. Soc. B 79, 144.10.1111/rssb.12233CrossRefGoogle ScholarPubMed
Carter, B. D. and Springer, M. D. (1977). The distribution of products, quotients and powers of independent H-function variates. SIAM J. Appl. Math. 33, 542558.10.1137/0133036CrossRefGoogle Scholar
Crane, H (2016). The ubiquitous Ewens sampling formula. Statist. Sci. 31, 119.Google Scholar
Curien, N. and Haas, B. (2013). The stable trees are nested. Prob. Theory Relat. Fields 157, 847883.10.1007/s00440-012-0472-xCrossRefGoogle Scholar
De Blasi, P., Favaro, S., Lijoi, A., Mena, R., Prünster, I. and Ruggiero, M. (2015). Are Gibbs-type priors the most natural generalization of the Dirichlet process? IEEE Trans. Pattern Anal. Mach. Intell. 37, 212229.10.1109/TPAMI.2013.217CrossRefGoogle ScholarPubMed
Dong, R., Goldschmidt, C. and Martin, J. (2006). Coagulation–fragmentation duality, Poisson–Dirichlet distributions and random recursive trees. Ann. Appl. Prob. 16, 17331750.10.1214/105051606000000655CrossRefGoogle Scholar
Ferguson, T. S. (1973). A Bayesian analysis of some nonparametric problems. Ann. Statist. 1, 209230.10.1214/aos/1176342360CrossRefGoogle Scholar
Gasthaus, J. and Teh, Y. W. (2010). Improvements to the sequence memoizer. In Proc. 24th Int. Conf. Neural Information Processing Systems, Vol. 1, eds. J. Lafferty, C. Williams, J. Shawe-Taylor, R. Zemel, and A. Culotta. ACM, pp. 685693.Google Scholar
Gnedin, A. and Pitman, J. (2006). Exchangeable Gibbs partitions and Stirling triangles. J. Math. Sci. 138, 56745685.10.1007/s10958-006-0335-zCrossRefGoogle Scholar
Gorenflo, R., Kilbas, A. A., Mainardi, F. and Rogosin, S. V. (2014). Mittag-Leffler Functions, Related Topics and Applications. Springer, Berlin.10.1007/978-3-662-43930-2CrossRefGoogle Scholar
Griffiths, R. C. and Spanò, D. (2007). Record indices and age-ordered frequencies in exchangeable Gibbs partitions. Electron. J. Prob. 12, 11011130.10.1214/EJP.v12-434CrossRefGoogle Scholar
Haas, B., Miermont, G., Pitman, J. and Winkel, M. (2008). Continuum tree asymptotics of discrete fragmentations and applications to phylogenetic models. Ann. Prob. 36, 17901837.10.1214/07-AOP377CrossRefGoogle Scholar
Haas, B., Pitman, J. and Winkel, M. (2009). Spinal partitions and invariance under re-rooting of continuum random trees. Ann. Prob. 37, 13811411.10.1214/08-AOP434CrossRefGoogle Scholar
Ho, M.-W., James, L. F. and Lau, J. W. (2007). Gibbs partitions (EPPF’s) derived from a stable subordinator are Fox H and Meijer G transforms. Preprint, arXiv:0708.0619 [math.PR].Google Scholar
Ho, M.-W., James, L. F. and Lau, J. W. (2021). Gibbs partitions, Riemann–Liouville fractional operators, Mittag-Leffler functions, and fragmentations derived from stable subordinators. J. Appl. Prob. 58, 314334.10.1017/jpr.2020.93CrossRefGoogle Scholar
Ishwaran, H. and James. L. F. (2001). Gibbs sampling methods for stick-breaking priors. J. Amer. Statist. Assoc. 96, 161173.10.1198/016214501750332758CrossRefGoogle Scholar
James, L. F. (2010). Lamperti type laws. Ann. Appl. Prob. 20, 13031340.10.1214/09-AAP660CrossRefGoogle Scholar
James, L. F. (2019). Stick-breaking Pitman–Yor processes given the species sampling size. Preprint, arXiv:1908.07186[math.ST].Google Scholar
James, L. F. (2022). Single-block recursive Poisson–Dirichlet fragmentations of normalized generalized Gamma processes. Mathematics 10, 561.10.3390/math10040561CrossRefGoogle Scholar
James, L. F, Lee, J. and Ross, N. (2024). Network and interaction models for data with hierarchical granularity via fragmentation and coagulation. Preprint, arXiv:2408.04866 [math.ST].Google Scholar
Lebedev, N. N. (1972). Special Functions and Their Applications. Dover Publications, New York.Google Scholar
Lomeli, M., Favaro, S. and Teh, Y. W. (2017). A marginal sampler for $\sigma$ -stable Poisson–Kingman mixture models. J. Comput. Graph. Statist. 26, 4453.Google Scholar
Mathai, A. M., Saxena, R. K. and Haubold, H. J. (2010). The H-Function. Theory and Applications. Springer, New York.Google Scholar
Pitman, J. (1996). Some developments of the Blackwell–MacQueen urn scheme. In Statistics, Probability and Game Theory (IMS Lect. Notes Monogr. Ser. 30). Institute of Mathematical Statistics, Hayward, CA, pp. 245267.10.1214/lnms/1215453576CrossRefGoogle Scholar
Pitman, J. (1999). Coalescents with multiple collisions. Ann. Prob. 27, 18701902.10.1214/aop/1022874819CrossRefGoogle Scholar
Pitman, J. (2003). Poisson–Kingman partitions. In Statistics and Science: A Festschrift for Terry Speed (IMS Lect. Notes Monogr. Ser. 40). Institute of Mathematical Statistics, Beachwood, OH, pp. 134.Google Scholar
Pitman, J. (2006). Combinatorial Stochastic Processes (Lect. Notes Math. 1875). Springer, Berlin.Google Scholar
Pitman, J. (2017). Mixed Poisson and negative binomial models for clustering and species sampling. Unpublished manuscript.Google Scholar
Pitman, J. and Yakubovich, Y. (2018). Ordered and size-biased frequencies in GEM and Gibbs models for species sampling. Ann. Appl. Prob. 28, 17931820.10.1214/17-AAP1343CrossRefGoogle Scholar
Pitman, J. and Yor, M. (1992). Arcsine laws and interval partitions derived from a stable subordinator. Proc. London Math. Soc. 65, 326356.10.1112/plms/s3-65.2.326CrossRefGoogle Scholar
Pitman, J. and Yor, M. (1997). The two-parameter Poisson–Dirichlet distribution derived from a stable subordinator. Ann. Prob. 25, 855900.10.1214/aop/1024404422CrossRefGoogle Scholar
Wood, F., Gasthaus, J., Archambeau, C., James, L. F. and Teh, Y. W. (2011). The sequence memoizer. Commun. ACM 54, 9198.10.1145/1897816.1897842CrossRefGoogle Scholar