|
| 1 | +<h1 align="center"> |
| 2 | + <br> |
| 3 | + <br> |
| 4 | + <img width="360" src="logo.png" alt="ulid"> |
| 5 | + <br> |
| 6 | + <br> |
| 7 | + <br> |
| 8 | +</h1> |
| 9 | + |
| 10 | +# Universally Unique Lexicographically Sortable Identifier |
| 11 | + |
| 12 | +UUID can be suboptimal for many uses-cases because: |
| 13 | + |
| 14 | +- It isn't the most character efficient way of encoding 128 bits of randomness |
| 15 | +- UUID v1/v2 is impractical in many environments, as it requires access to a unique, stable MAC address |
| 16 | +- UUID v3/v5 requires a unique seed and produces randomly distributed IDs, which can cause fragmentation in many data structures |
| 17 | +- UUID v4 provides no other information than randomness which can cause fragmentation in many data structures |
| 18 | + |
| 19 | +Instead, herein is proposed ULID: |
| 20 | + |
| 21 | +```javascript |
| 22 | +ulid() // 01ARZ3NDEKTSV4RRFFQ69G5FAV |
| 23 | +``` |
| 24 | + |
| 25 | +- 128-bit compatibility with UUID |
| 26 | +- 1.21e+24 unique ULIDs per millisecond |
| 27 | +- Lexicographically sortable! |
| 28 | +- Canonically encoded as a 26 character string, as opposed to the 36 character UUID |
| 29 | +- Uses Crockford's base32 for better efficiency and readability (5 bits per character) |
| 30 | +- Case insensitive |
| 31 | +- No special characters (URL safe) |
| 32 | +- Monotonic sort order (correctly detects and handles the same millisecond) |
| 33 | + |
| 34 | +## Implementations in other languages |
| 35 | + |
| 36 | +From ourselves and the community! |
| 37 | + |
| 38 | +| Language | Author | Binary Implementation | |
| 39 | +| -------- | ------ | --------------------- | |
| 40 | +| [C++](https://github.com/suyash/ulid) | [suyash](https://github.com/suyash) | ✓ | |
| 41 | +| [Clojure](https://github.com/theikkila/clj-ulid) | [theikkila](https://github.com/theikkila) | |
| 42 | +| [Objective-C](https://github.com/whitesmith/ulid) | [ricardopereira](https://github.com/ricardopereira) | |
| 43 | +| [Crystal](https://github.com/SuperPaintman/ulid) | [SuperPaintman](https://github.com/SuperPaintman) | |
| 44 | +| [Dart](https://github.com/agilord/ulid) | [isoos](https://github.com/isoos) | ✓ | |
| 45 | +| [Delphi](https://github.com/martinusso/ulid) | [matinusso](https://github.com/martinusso) | |
| 46 | +| [D](https://github.com/enckse/ulid) | [enckse](https://github.com/enckse) | |
| 47 | +| [D (dub)](https://code.dlang.org/packages/ulid-d) | [extrawurst](https://github.com/Extrawurst) |
| 48 | +| [Erlang](https://github.com/savonarola/ulid) | [savonarola](https://github.com/savonarola) | |
| 49 | +| [Elixir](https://github.com/merongivian/ulid) | [merongivian](https://github.com/merongivian) | |
| 50 | +| [F#](https://github.com/lucasschejtman/FSharp.Ulid) | [lucasschejtman](https://github.com/lucasschejtman) | |
| 51 | +| [Go](https://github.com/imdario/go-ulid) | [imdario](https://github.com/imdario/) | |
| 52 | +| [Go](https://github.com/oklog/ulid) | [oklog](https://github.com/oklog) | ✓ | |
| 53 | +| [Haskell](https://github.com/steven777400/ulid) | [steven777400](https://github.com/steven777400) | ✓ | |
| 54 | +| [Java](https://github.com/huxi/sulky/tree/master/sulky-ulid) | [huxi](https://github.com/huxi) | ✓ | |
| 55 | +| [Java](https://github.com/azam/ulidj) | [azam](https://github.com/azam) | |
| 56 | +| [Java](https://github.com/Lewiscowles1986/jULID) | [Lewiscowles1986](https://github.com/Lewiscowles1986) | |
| 57 | +| [JavaScript](https://github.com/ulid/javascript) | [ulid](https://github.com/ulid) | |
| 58 | +| [Julia](https://github.com/ararslan/ULID.jl) | [ararslan](https://github.com/ararslan) | |
| 59 | +| [Lua](https://github.com/Tieske/ulid.lua) | [Tieske](https://github.com/Tieske) | |
| 60 | +| [.NET](https://github.com/RobThree/NUlid) | [RobThree](https://github.com/RobThree) | ✓ | |
| 61 | +| [.NET](https://github.com/fvilers/ulid.net) | [fvilers](https://github.com/fvilers) |
| 62 | +| [Nim](https://github.com/adelq/ulid) | [adelq](https://github.com/adelq) |
| 63 | +| [Perl 5](https://github.com/bk/Data-ULID) | [bk](https://github.com/bk) | ✓ | |
| 64 | +| [PHP](https://github.com/Lewiscowles1986/ulid) | [Lewiscowles1986](https://github.com/Lewiscowles1986) | |
| 65 | +| [PHP](https://github.com/robinvdvleuten/php-ulid) | [robinvdvleuten](https://github.com/robinvdvleuten) | |
| 66 | +| [PowerShell](https://github.com/PetterBomban/posh-ulid) | [PetterBomban](https://github.com/PetterBomban) | |
| 67 | +| [Python](https://github.com/mdipierro/ulid) | [mdipierro](https://github.com/mdipierro) | |
| 68 | +| [Python](https://github.com/ahawker/ulid) | [ahawker](https://github.com/ahawker) | ✓ | |
| 69 | +| [Python](https://github.com/mdomke/python-ulid) | [mdomke](https://github.com/mdomke) | ✓ | |
| 70 | +| [Ruby](https://github.com/rafaelsales/ulid) | [rafaelsales](https://github.com/rafaelsales) | |
| 71 | +| [Rust](https://github.com/mmacedoeu/rulid.rs) | [mmacedoeu](https://github.com/mmacedoeu/rulid.rs) | ✓ | |
| 72 | +| [Rust](https://github.com/dylanhart/ulid-rs) | [dylanhart](https://github.com/dylanhart) | ✓ | |
| 73 | +| [SQL-Microsoft](https://github.com/rmalayter/ulid-mssql) | [rmalayter](https://github.com/rmalayter) | ✓ | |
| 74 | +| [Swift](https://github.com/simonwhitehouse/ULIDSwift) | [simonwhitehouse](https://github.com/simonwhitehouse) | |
| 75 | +| [Tcl](https://wiki.tcl-lang.org/48827) | [dbohdan](https://github.com/dbohdan) | |
| 76 | + |
| 77 | +## Specification |
| 78 | + |
| 79 | +Below is the current specification of ULID as implemented in this repository. |
| 80 | + |
| 81 | +*Note: the binary format has not been implemented in JavaScript as of yet.* |
| 82 | + |
| 83 | +``` |
| 84 | + 01AN4Z07BY 79KA1307SR9X4MV3 |
| 85 | +
|
| 86 | +|----------| |----------------| |
| 87 | + Timestamp Randomness |
| 88 | + 48bits 80bits |
| 89 | +``` |
| 90 | + |
| 91 | +### Components |
| 92 | + |
| 93 | +**Timestamp** |
| 94 | +- 48 bit integer |
| 95 | +- UNIX-time in milliseconds |
| 96 | +- Won't run out of space till the year 10895 AD. |
| 97 | + |
| 98 | +**Randomness** |
| 99 | +- 80 bits |
| 100 | +- Cryptographically secure source of randomness, if possible |
| 101 | + |
| 102 | +### Sorting |
| 103 | + |
| 104 | +The left-most character must be sorted first, and the right-most character sorted last (lexical order). The default ASCII character set must be used. Within the same millisecond, sort order is not guaranteed |
| 105 | + |
| 106 | +### Canonical String Representation |
| 107 | + |
| 108 | +``` |
| 109 | +ttttttttttrrrrrrrrrrrrrrrr |
| 110 | +
|
| 111 | +where |
| 112 | +t is Timestamp (10 characters) |
| 113 | +r is Randomness (16 characters) |
| 114 | +``` |
| 115 | + |
| 116 | +#### Encoding |
| 117 | + |
| 118 | +Crockford's Base32 is used as shown. This alphabet excludes the letters I, L, O, and U to avoid confusion and abuse. |
| 119 | + |
| 120 | +``` |
| 121 | +0123456789ABCDEFGHJKMNPQRSTVWXYZ |
| 122 | +``` |
| 123 | + |
| 124 | +### Monotonicity |
| 125 | + |
| 126 | +When generating a ULID within the same millisecond, we can provide some |
| 127 | +guarantees regarding sort order. Namely, if the same millisecond is detected, the `random` component is incremented by 1 bit in the least significant bit position (with carrying). For example: |
| 128 | + |
| 129 | +```javascript |
| 130 | +import { monotonicFactory } from 'ulid' |
| 131 | + |
| 132 | +const ulid = monotonicFactory() |
| 133 | + |
| 134 | +// Assume that these calls occur within the same millisecond |
| 135 | +ulid() // 01BX5ZZKBKACTAV9WEVGEMMVRZ |
| 136 | +ulid() // 01BX5ZZKBKACTAV9WEVGEMMVS0 |
| 137 | +``` |
| 138 | + |
| 139 | +If, in the extremely unlikely event that, you manage to generate more than 2<sup>80</sup> ULIDs within the same millisecond, or cause the random component to overflow with less, the generation will fail. |
| 140 | + |
| 141 | +```javascript |
| 142 | +import { monotonicFactory } from 'ulid' |
| 143 | + |
| 144 | +const ulid = monotonicFactory() |
| 145 | + |
| 146 | +// Assume that these calls occur within the same millisecond |
| 147 | +ulid() // 01BX5ZZKBKACTAV9WEVGEMMVRY |
| 148 | +ulid() // 01BX5ZZKBKACTAV9WEVGEMMVRZ |
| 149 | +ulid() // 01BX5ZZKBKACTAV9WEVGEMMVS0 |
| 150 | +ulid() // 01BX5ZZKBKACTAV9WEVGEMMVS1 |
| 151 | +... |
| 152 | +ulid() // 01BX5ZZKBKZZZZZZZZZZZZZZZX |
| 153 | +ulid() // 01BX5ZZKBKZZZZZZZZZZZZZZZY |
| 154 | +ulid() // 01BX5ZZKBKZZZZZZZZZZZZZZZZ |
| 155 | +ulid() // throw new Error()! |
| 156 | +``` |
| 157 | + |
| 158 | +#### Overflow Errors when Parsing Base32 Strings |
| 159 | + |
| 160 | +Technically, a 26-character Base32 encoded string can contain 130 bits of information, whereas a ULID must only contain 128 bits. Therefore, the largest valid ULID encoded in Base32 is `7ZZZZZZZZZZZZZZZZZZZZZZZZZ`, which corresponds to an epoch time of `281474976710655` or `2 ^ 48 - 1`. |
| 161 | + |
| 162 | +Any attempt to decode or encode a ULID larger than this should be rejected by all implementations, to prevent overflow bugs. |
| 163 | + |
| 164 | +### Binary Layout and Byte Order |
| 165 | + |
| 166 | +The components are encoded as 16 octets. Each component is encoded with the Most Significant Byte first (network byte order). |
| 167 | + |
| 168 | +``` |
| 169 | +0 1 2 3 |
| 170 | + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 |
| 171 | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 172 | +| 32_bit_uint_time_high | |
| 173 | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 174 | +| 16_bit_uint_time_low | 16_bit_uint_random | |
| 175 | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 176 | +| 32_bit_uint_random | |
| 177 | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 178 | +| 32_bit_uint_random | |
| 179 | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| 180 | +``` |
| 181 | + |
| 182 | +## Prior Art |
| 183 | + |
| 184 | +Partly inspired by: |
| 185 | +- http://instagram-engineering.tumblr.com/post/10853187575/sharding-ids-at-instagram |
| 186 | +- https://firebase.googleblog.com/2015/02/the-2120-ways-to-ensure-unique_68.html |
0 commit comments