sysl.encoding

Every conversion here is total in one direction and fallible in the other — encoding cannot fail, decoding can — so the decoders answer a Result carrying DecodeError, which says why at the granularity a caller can act on rather than only that something was wrong.

None of it interprets what it moves. A base64 decoder does not know whether the bytes it produced are a key, an image or a sentence, and a u32 read out of four bytes is a number and not a field of anything. What the bytes mean belongs to whatever asked for them.

Index

nil base64_decode base64_decoded_len base64_encode base64_string get_u16_be get_u16_le get_u32_be get_u32_le get_u64_be get_u64_le hex_decode hex_decoded_len hex_encode hex_string of_bytes parse percent_decode percent_decode_string percent_decoded_len percent_encode percent_string push_u16_be push_u16_le push_u32_be push_u32_le push_u64_be push_u64_le put_u16_be put_u16_le put_u32_be put_u32_le put_u64_be put_u64_le to_string v4 v4_of v7 Alphabet DecodeError Percent Uuid Plain Display for DecodeError Display for Uuid

Values

nil

val nil: Uuid = Uuid(…)

The UUID that names nothing: sixteen zero bytes, 00000000-0000-0000-0000-000000000000.

Its version and variant both read as zero, which is not a mistake in the constant – RFC 9562 exempts it, and a reader checking version() == 4 on an unset field is meant to see that it is unset rather than to see a well-formed value.

Functions

base64_decode

base64_decode(text: []const u8, out: []u8) -> Result[usize, DecodeError]

The bytes the text spells, written into out, and how many there were.

Six bits at a time into an accumulator, rather than a four-character group at a time, because the unpadded spellings end mid-group and a group-at-a-time reader needs a special case for each of them. Whole bytes fall out of the accumulator as they complete, and the two or four leftover bits at the end are the padding bits, which are discarded.

base64_decoded_len

base64_decoded_len(text: []const u8) -> usize

How many bytes base64_decode will write, so a caller can size the slice it lends. Trailing padding is discounted, so this answers the same number for the padded and unpadded spellings of one input.

base64_encode

base64_encode(bytes: []const u8, out: *Writer, alphabet: Alphabet, pad: bool)

Three bytes become four characters. The tail – one or two bytes left over – becomes two or three characters, padded out to four with = when padding was asked for.

base64_string

base64_string(bytes: []const u8, alphabet: Alphabet, pad: bool) -> string

The same, gathered into a string. Every character emitted is ASCII, so the bytes are valid UTF-8 by construction.

get_u16_be

get_u16_be(b: []const u8) -> Option[u16]

get_u16_le

get_u16_le(b: []const u8) -> Option[u16]

get_u32_be

get_u32_be(b: []const u8) -> Option[u32]

get_u32_le

get_u32_le(b: []const u8) -> Option[u32]

get_u64_be

get_u64_be(b: []const u8) -> Option[u64]

get_u64_le

get_u64_le(b: []const u8) -> Option[u64]

hex_decode

hex_decode(text: []const u8, out: []u8) -> Result[usize, DecodeError]

The bytes the text spells, written into out, and how many there were.

hex_decoded_len

hex_decoded_len(text: []const u8) -> usize

How many bytes hex_decode will write, so a caller can size the slice it lends.

Answers for the length alone and does not look at the content: a text of odd length has no whole number of bytes in it and hex_decode refuses it, but the halving is still the right size to have reserved.

hex_encode

hex_encode(bytes: []const u8, out: *Writer)

Two characters per byte, written straight through to the sink.

hex_string

hex_string(bytes: []const u8) -> string

The same, gathered into a string. Every byte emitted above is an ASCII digit, so the result is valid UTF-8 by construction and needs no validating pass.

of_bytes

of_bytes(b: [16]u8) -> Uuid

A UUID over bytes the caller already has – read off a wire, out of a row, out of a file.

It sets no bits and checks none. What arrives is what the value holds, so a UUID from somewhere else survives a round trip through this module unchanged, including one whose variant is Microsoft’s or whose version is a number this library does not generate. version and variant are how a caller asks what it got.

parse

parse(s: string) -> Option[Uuid]

The 36-character form read back, or None.

Only that form. Braces, a urn:uuid: prefix and the unhyphenated 32-character form are all refused, because accepting them makes parse a guess about which of several conventions the text follows – and a caller that knows which one it is reading can strip a wrapper in a line. Either case of hex digit is accepted, which is what RFC 9562 asks for.

A hyphen in the wrong place is refused as firmly as a bad digit: the positions are what tell this form from the others, so a text with them elsewhere is a different format rather than a sloppy spelling of this one.

percent_decode

percent_decode(text: []const u8, out: []u8, mode: Percent) -> Result[usize, DecodeError]

The bytes the text spells, written into out, and how many there were.

A malformed escape is refused, never passed through: a % with fewer than two bytes after it is BadLength, and a % followed by a byte that is not a hex digit is BadByte at that byte’s offset. Either case of hex digit is accepted. Any other byte is copied as it stands, so decoding does not check that the output is UTF-8 – what the bytes mean is the caller’s.

percent_decode_string

percent_decode_string(text: []const u8, mode: Percent) -> Result[string, DecodeError]

The text the percent-encoded text spells: percent_decode, then a UTF-8 check on the result.

A query value or a path segment is text, so this is what most callers want. The check is on the decoded bytes, which is where an escape can make invalid text out of valid: %C3%28 is a lead byte followed by (. That is Utf8(at) with at the offset in the decoded output; the escape errors are percent_decode‘s. Allocates the result.

percent_decoded_len

percent_decoded_len(text: []const u8) -> usize

How many bytes percent_decode will write for a well-formed text: every %XX is one byte and every other byte is itself. A truncated escape is not counted as anything; percent_decode refuses it.

percent_encode

percent_encode(bytes: []const u8, out: *Writer, mode: Percent)

Each byte of bytes as itself where unreserved and as %XX (upper-case hex, as RFC 3986 recommends) otherwise. Works on bytes, so UTF-8 text is escaped one byte of each multibyte character at a time.

percent_string

percent_string(bytes: []const u8, mode: Percent) -> string

The same, gathered into a string. Every byte emitted is ASCII, so the result is valid UTF-8.

push_u16_be

push_u16_be(out: Buf[u8], v: u16)

push_u16_le

push_u16_le(out: Buf[u8], v: u16)

push_u32_be

push_u32_be(out: Buf[u8], v: u32)

push_u32_le

push_u32_le(out: Buf[u8], v: u32)

push_u64_be

push_u64_be(out: Buf[u8], v: u64)

push_u64_le

push_u64_le(out: Buf[u8], v: u64)

put_u16_be

put_u16_be(b: []u8, v: u16) -> bool

put_u16_le

put_u16_le(b: []u8, v: u16) -> bool

put_u32_be

put_u32_be(b: []u8, v: u32) -> bool

put_u32_le

put_u32_le(b: []u8, v: u32) -> bool

put_u64_be

put_u64_be(b: []u8, v: u64) -> bool

put_u64_le

put_u64_le(b: []u8, v: u64) -> bool

to_string

to_string(u: Uuid) -> string

The 36-character form, lowercase, hyphenated 8-4-4-4-12.

Lowercase because RFC 9562 says to emit lowercase and accept either, and parse accepts either. This is the only thing in the module that allocates, and it allocates one 36-byte string.

v4

v4(rng: *Rng) -> Uuid

A version 4 UUID: 122 random bits, with the six that name the version and the variant written over them.

Rng IS NOT UNPREDICTABLE, AND THAT DECIDES WHAT THIS MAY BE USED FOR. sysl.rand is PCG32 – fast, seedable, reproducible, and completely determined by its state, so anyone who sees a handful of ids can compute every id that follows. That is fine for a primary key, a request id, a correlation id in a log: things that must be distinct. It is not fine for a session token, a password-reset link, an object name that stands in for an access check, or anything else that must be unguessable.

For those, take the bytes from the kernel and use v4_of – sysl.posix.rand.entropy_from_os is exactly that source, and the module comment there draws the same line for the same reason. Seeding a predictable generator unpredictably does not make it unpredictable.

The generator is a parameter rather than something this module keeps, so nothing here is module storage and a program on a board can supply one it seeded itself.

v4_of

v4_of(b: [16]u8) -> Uuid

A version 4 UUID over sixteen bytes the caller brings, with the version and variant bits written over them.

This is the one to use where the value must be unguessable, since what it is worth is exactly what the bytes are worth. entropy_from_os fills a slice straight from the kernel’s pool:

var raw: [16]u8

if entropy_from_os(raw[0..<16]) then print(v4_of(raw))

Six of the 128 bits are overwritten, which is the version and the variant and is what makes it a v4 rather than sixteen anonymous bytes. A caller wanting all 128 to be its own wants of_bytes and does not want a version number.

v7

v7(t: Instant, rng: *Rng) -> Uuid

A version 7 UUID: the instant in the high 48 bits as milliseconds since the Unix epoch, and 74 random bits under it.

What it buys is that ids made in order sort in order, as bytes and as text, which is what a database index wants from a primary key and what a v4 conspicuously does not give. Two ids made in the same millisecond are ordered by their random bits, which is to say arbitrarily – v7 promises ordering between milliseconds and nothing finer.

The instant is a parameter, so this module reaches no clock. A host passes sysl.time.now(); a board passes whatever its own counter says, and a board with no notion of the epoch should be making v4s instead, since a timestamp that is not one is a field that lies.

The same warning v4 carries applies to the random half: a v7 built from sysl.rand is not unguessable, and it is a good deal more predictable than a v4, since 48 of its bits are a timestamp anybody can compute.

48 bits of milliseconds runs out in the year 10889, so the wrap is not a thing to plan for. An instant before the epoch has a negative millisecond count and is truncated to its low 48 bits, which is a value that does not order correctly – v7 has no way to say “before 1970” and this does not invent one.

Types

Alphabet

enum Alphabet
    Standard
    UrlSafe

Which of RFC 4648’s two alphabets. They differ only in the last two characters: +/ reads badly in a URL and in a filename, which is the whole reason the second exists.

DecodeError

enum DecodeError
    BadByte(at: usize)
    BadLength
    BadPadding(at: usize)
    Short(needed: usize)
    Utf8(at: usize)

Why a decode refused, at the granularity a caller can act on.

A type of its own rather than sysl.text‘s ParseError, which is about reading a number out of text. Two of that type’s four variants – Overflow and BadBase – can never occur here, and a caller matching on it would be told cases exist that cannot. The shape is copied deliberately: the variants are separated by what a caller would do about each and not by taxonomy.

BadByte and BadPadding carry the offset for the reason ParseError.BadDigit and Utf8Error do – a message naming where is worth writing and cannot be reconstructed afterwards. Short carries the length that would have been enough, Utf8 the offset in the decoded bytes where text stopped being valid UTF-8 (only the string-producing decoders say it), so a caller can resize and retry in one step rather than discovering the requirement one byte at a time.

Percent

enum Percent
    Component
    Form

Which dialect of percent-encoding. The two differ in exactly one byte, the space.

Uuid

struct Uuid
    private raw: [16]u8

Sixteen bytes with a layout: what the version is, what the variant is, and – for a v7 – when it was made.

The bytes are held in the order they are written, which is the order RFC 9562 numbers them and is big-endian throughout. That is what makes to_bytes directly comparable with a UUID from anywhere else, and what makes a v7 sort by time as a plain byte string.

Ord for Uuid is therefore over those bytes in order, which for a v7 is over the timestamp first – the whole point of that version. For a v4 it is an arbitrary total order, which is what a sorted container wants and means nothing else.

All three traits are DERIVED, which they could not be until an array had Ord and Hash. The one field is a [16]u8 and an array’s ordering is lexicographic, first element first – so the derived comparison is byte order, which is exactly what this type wants and what the two hand-written blocks here used to spell out. Hash is FNV-1a over the elements, and a UUID’s bytes are already well distributed, so all a hash has to add is that a table bucketing on the low bits sees all of them.

Member	Signature	Description
to_bytes	to_bytes(self) -> [16]u8	The sixteen bytes, in RFC order.
version	version(self) -> u8	Which of RFC 9562’s versions this claims to be: the high four bits of byte 6.
variant	variant(self) -> u8	Which family of layouts byte 8 puts this in – 1 for RFC 9562’s, which is what everything generated here is, and what any UUID a reader is likely to meet is.
is_nil	is_nil(self) -> bool	Whether every byte is zero – the nil UUID, which RFC 9562 defines as the value that names nothing.

Traits

Plain

trait Plain
    decode(b: []const u8) -> Option[Self]
    encode(self, out: []u8) -> bool

A type whose value is its bytes and nothing else, read out of them and written back.

The membership is the compiler’s, as sysl.math.Bits‘ is: a type is plain where every part of it is an integer of whole bytes, a float, a fixed-point number, a bool, a simple enum, or an array or a struct of plain things. Nothing is written to join – a superblock, an inode or a protocol header declared as an ordinary struct already is one. What is shut out is whatever means something only inside the running program: a pointer, a counted reference, a string, a slice. A char and a subtype with a range are out too, their rule being about values rather than bytes; hold the number and convert it, which runs the check.

decode is the checked direction. It answers None where the slice is shorter than the type, and where a byte that could be wrong is: a bool holding anything but 0 or 1, an enum holding a number none of its variants is. Every other byte is some value of its field, so there is nothing to check there. It reads the first sizeof(T) bytes and leaves the rest – a header at the front of a frame is the ordinary case – and needs no alignment of the slice.

encode writes zeros into padding, so one value always encodes to the same bytes. It answers false, and writes nothing, where the slice is shorter than the type.

The byte order is the struct’s. An integer field is written in the machine’s order unless the struct says otherwise with @byte_order(big) or @byte_order(little), and then it is in that order on every machine (reference/attributes.md § @byte_order).

Member	Signature	Description
decode	decode(b: []const u8) -> Option[Self]	The value the front of b holds, or None where b is short or holds a byte no value of the type has.
encode	encode(self, out: []u8) -> bool	This value’s bytes at the front of out, or false where out is too short to hold them.

Implementations

Display for DecodeError

impl Display for DecodeError

The rendering, so that a refusal can be printed without matching on it.

Display for Uuid

impl Display for Uuid