Replace block module with helper functions

[newpavlov]

This change allows to remove the serde feature and reduce total amount of code in the crate. Additionally, the helper block functions now store the cursor position in the first buffer value, which allows us to remove the separate index field.

The new examples demonstrate implementation of the traits for the most common RNG classes.

TODO:

• Check value stability in downstream impls.
• Write integration tests with 100% coverage.

[dhardy]

What does this look like at the use site? You could update rand_chacha at least (same repo; yes I am going to have to remake those PRs to move it to the rngs repo anyway).

[dhardy]

Motivation regarding serde is good; the feature does not really belong in this crate.

lazy block generation

Not a change.

The le module is renamed to utils

I explained why I didn't want to do this in rust-random/rand#1667.

[newpavlov]

What does this look like at the use site?

See the new doc examples. I wanted to update the implementation crates after I got a confirmation that you approve the new approach.

Not a change.

True. I will edit the OP.

I explained why I didn't want to do this

The problem is that the new_buffer function and the Word trait technically have nothing to do with "litttle endian". And I think utils::fill_bytes_via_gen_block looks better in user code than le::fill_bytes_via_gen_block. But it's not a strong opinion, so I can rename it back, if you insist.

[newpavlov]

All potential panics in fill_bytes_via_gen_block get eliminated successfully: https://rust.godbolt.org/z/6E9Y7vW3o Unfortunately, codegen is a bit worse than expected. The compiler unrolls the head and tail processing loops instead of compiling them into one memcpy. I will look into whether it's possible to tweak the code to prevent it.

next_word_via_gen_block looks good: https://rust.godbolt.org/z/5qd7MaKx3 Same for fill_bytes_via_next_word: https://rust.godbolt.org/z/esYo49oKK

[newpavlov]

I improved codegen for fill_bytes_via_gen_block (see https://rust.godbolt.org/z/T8f77KjGc), but unfortunately it involves unsafe code. In future it could be removed with generic_const_exprs, but for now I couldn't find a way to do it on Rust 1.85 without returning to the bad codegen.

[newpavlov]

It's fine to put those in a new utils module then.

I really do not want to introduce two separate utility modules. I returned the le name for now.

Isn't the best validation of a proof-of-concept API to use it?

I did use them in the examples. I plan to update chacha20 a bit later.

[dhardy]

I really do not want to introduce two separate utility modules.

Better to punt the issue to a new PR than to worry overly about it here. More important that this PR is easy to review.

I did use them in the examples.

Yes, but examples don't always match up with real usage, so I strongly prefer seeing a new API in a real use-case than just in an example. Doc examples are also harder to review (no highlighting without building docs).

[dhardy]

The approach looks broadly okay, not losing much from our previous code. But I cannot approve it without running benchmarks on actual block RNGs, so it does need some downstream implementation before this PR can be approved.

[newpavlov]

I migrated the rngs repository to this PR, see: rust-random/rngs#78

The only encountered problem is with implementation of next_u32 for RNGs which output [u64; N] . The current implementation pops u64 from the buffer and truncates it to 32 bits. Meanwhile the old implementation tracks halves. I guess it means that we need a separate helper function for this case.

Migration of chacha20 will take more time, since it needs to be refactored a fair bit.

[dhardy]

Another potential issue is ReseedingRng. This is the only thing I am aware of using the cores of block RNGs directly. I think the primary motivation is to limit the reseed check to block generation.

[newpavlov]

Considering that reseeding should be done using relatively large thresholds, buffer re-initialization is an extremely cheap operation (in practice the additional writes are unlikely to even leave the CPU cache before getting overwritten by a newly generated block), so I don't think it's a deal breaker.

[dhardy]

The point was more to avoid needing to test whether to reseed frequently. It may not be necessary still (testing required), but direct access to a block generator would allow a cleaner design of ReseedingRng.