Optimum Achievable Rates in Two Random Number Generation Problems with <i>f</i>-Divergences Using Smooth Rényi Entropy
Two typical fixed-length random number generation problems in information theory are considered for <i>general</i> sources. One is the source resolvability problem and the other is the intrinsic randomness problem. In each of these problems, the optimum achievable rate with respect to th...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-09-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/26/9/766 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850261061932941312 |
|---|---|
| author | Ryo Nomura Hideki Yagi |
| author_facet | Ryo Nomura Hideki Yagi |
| author_sort | Ryo Nomura |
| collection | DOAJ |
| description | Two typical fixed-length random number generation problems in information theory are considered for <i>general</i> sources. One is the source resolvability problem and the other is the intrinsic randomness problem. In each of these problems, the optimum achievable rate with respect to the given approximation measure is one of our main concerns and has been characterized using two different information quantities: the information spectrum and the smooth Rényi entropy. Recently, optimum achievable rates with respect to <i>f</i>-divergences have been characterized using the information spectrum quantity. The <i>f</i>-divergence is a general non-negative measure between two probability distributions on the basis of a convex function <i>f</i>. The class of <i>f</i>-divergences includes several important measures such as the variational distance, the KL divergence, the Hellinger distance and so on. Hence, it is meaningful to consider the random number generation problems with respect to <i>f</i>-divergences. However, optimum achievable rates with respect to <i>f</i>-divergences using the smooth Rényi entropy have not been clarified yet in both problems. In this paper, we try to analyze the optimum achievable rates using the smooth Rényi entropy and to extend the class of <i>f</i>-divergence. To do so, we first derive general formulas of the <i>first-order</i> optimum achievable rates with respect to <i>f</i>-divergences in both problems under the same conditions as imposed by previous studies. Next, we relax the conditions on <i>f</i>-divergence and generalize the obtained general formulas. Then, we particularize our general formulas to several specified functions <i>f</i>. As a result, we reveal that it is easy to derive optimum achievable rates for several important measures from our general formulas. Furthermore, a kind of <i>duality</i> between the resolvability and the intrinsic randomness is revealed in terms of the smooth Rényi entropy. <i>Second-order</i> optimum achievable rates and optimistic achievable rates are also investigated. |
| format | Article |
| id | doaj-art-4cbd80ccf92740b9adefd769e719fc2d |
| institution | OA Journals |
| issn | 1099-4300 |
| language | English |
| publishDate | 2024-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj-art-4cbd80ccf92740b9adefd769e719fc2d2025-08-20T01:55:31ZengMDPI AGEntropy1099-43002024-09-0126976610.3390/e26090766Optimum Achievable Rates in Two Random Number Generation Problems with <i>f</i>-Divergences Using Smooth Rényi EntropyRyo Nomura0Hideki Yagi1Center for Data Science, Waseda University, Tokyo 169-8050, JapanDepartment of Computer and Network Engineering, The University of Electro-Communications, Tokyo 182-8585, JapanTwo typical fixed-length random number generation problems in information theory are considered for <i>general</i> sources. One is the source resolvability problem and the other is the intrinsic randomness problem. In each of these problems, the optimum achievable rate with respect to the given approximation measure is one of our main concerns and has been characterized using two different information quantities: the information spectrum and the smooth Rényi entropy. Recently, optimum achievable rates with respect to <i>f</i>-divergences have been characterized using the information spectrum quantity. The <i>f</i>-divergence is a general non-negative measure between two probability distributions on the basis of a convex function <i>f</i>. The class of <i>f</i>-divergences includes several important measures such as the variational distance, the KL divergence, the Hellinger distance and so on. Hence, it is meaningful to consider the random number generation problems with respect to <i>f</i>-divergences. However, optimum achievable rates with respect to <i>f</i>-divergences using the smooth Rényi entropy have not been clarified yet in both problems. In this paper, we try to analyze the optimum achievable rates using the smooth Rényi entropy and to extend the class of <i>f</i>-divergence. To do so, we first derive general formulas of the <i>first-order</i> optimum achievable rates with respect to <i>f</i>-divergences in both problems under the same conditions as imposed by previous studies. Next, we relax the conditions on <i>f</i>-divergence and generalize the obtained general formulas. Then, we particularize our general formulas to several specified functions <i>f</i>. As a result, we reveal that it is easy to derive optimum achievable rates for several important measures from our general formulas. Furthermore, a kind of <i>duality</i> between the resolvability and the intrinsic randomness is revealed in terms of the smooth Rényi entropy. <i>Second-order</i> optimum achievable rates and optimistic achievable rates are also investigated.https://www.mdpi.com/1099-4300/26/9/766<i>f</i>-divergenceHellinger distanceintrinsic randomnessKullback–Leibler divergencerandom number generationsmooth Rényi entropy |
| spellingShingle | Ryo Nomura Hideki Yagi Optimum Achievable Rates in Two Random Number Generation Problems with <i>f</i>-Divergences Using Smooth Rényi Entropy Entropy <i>f</i>-divergence Hellinger distance intrinsic randomness Kullback–Leibler divergence random number generation smooth Rényi entropy |
| title | Optimum Achievable Rates in Two Random Number Generation Problems with <i>f</i>-Divergences Using Smooth Rényi Entropy |
| title_full | Optimum Achievable Rates in Two Random Number Generation Problems with <i>f</i>-Divergences Using Smooth Rényi Entropy |
| title_fullStr | Optimum Achievable Rates in Two Random Number Generation Problems with <i>f</i>-Divergences Using Smooth Rényi Entropy |
| title_full_unstemmed | Optimum Achievable Rates in Two Random Number Generation Problems with <i>f</i>-Divergences Using Smooth Rényi Entropy |
| title_short | Optimum Achievable Rates in Two Random Number Generation Problems with <i>f</i>-Divergences Using Smooth Rényi Entropy |
| title_sort | optimum achievable rates in two random number generation problems with i f i divergences using smooth renyi entropy |
| topic | <i>f</i>-divergence Hellinger distance intrinsic randomness Kullback–Leibler divergence random number generation smooth Rényi entropy |
| url | https://www.mdpi.com/1099-4300/26/9/766 |
| work_keys_str_mv | AT ryonomura optimumachievableratesintworandomnumbergenerationproblemswithifidivergencesusingsmoothrenyientropy AT hidekiyagi optimumachievableratesintworandomnumbergenerationproblemswithifidivergencesusingsmoothrenyientropy |