Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add icu4x_shared with some helpers #5101

Draft
wants to merge 7 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 142 additions & 0 deletions components/properties/icu4x_shared.rs
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why wouldn't Transparent live in zerovec? There doesn't seem to be a use for it outside of an easier VarULE implementation.

Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
// This file is part of ICU4X. For terms of use, please see the file
// called LICENSE at the top level of the ICU4X source tree
// (online at: https://github.com/unicode-org/icu4x/blob/main/LICENSE ).

/// Trait asserting that a type is `repr(transparent)`. Used as a bound
/// for functions that require this invariant.
///
/// # Safety
///
/// 1. This outer type *must* be `repr(transparent)` over `Inner`
/// 2. `validate_inner` *must* return `false` if `Inner` does not uphold
/// invariants enforced by this outer type.
pub(crate) unsafe trait Transparent<Inner: ?Sized> {
#[allow(dead_code)]
fn validate_inner(inner: &Inner) -> bool;
#[allow(dead_code)]
fn as_inner(&self) -> &Inner;
}

/// Implements private helper functions for `repr(transparent)` types.
#[allow(unused_macros)]
macro_rules! impl_transparent_helpers {
($outer:ident($inner:path)) => {
impl $outer {
/// Casts the inner type to the outer type.
///
/// This function is safe, but it does not validate invariants
/// that the outer type might enforce. It is made available as
/// a private fn which can be called by another fn.
#[allow(dead_code)]
const fn cast_ref_unchecked(inner: &$inner) -> &$outer
where
$outer: Transparent<$inner>,
{
// Safety: Outer is repr(transparent) over Inner
// (enforced via trait safety invariant)
unsafe { &*(inner as *const $inner as *const $outer) }
}
/// Casts the inner type to the outer type.
///
/// This function is safe, but it does not validate invariants
/// that the outer type might enforce. It is made available as
/// a private fn which can be called by another fn.
#[allow(dead_code)]
const fn cast_box_unchecked(
inner: alloc::boxed::Box<$inner>,
) -> alloc::boxed::Box<$outer>
where
$outer: Transparent<$inner>,
{
// Safety: Outer is repr(transparent) over Inner
// (enforced via trait safety invariant)
unsafe { core::mem::transmute(inner) }
}
}
};
}

/// Implements `VarULE` on a `repr(transparent)` type.
#[allow(unused_macros)]
macro_rules! impl_transparent_varule {
($outer:ident($inner:path)) => {
// Safety:
//
// 1. `repr(transparent)`, enforced by trait bound, implies no padding
// 2. `repr(transparent)`, enforced by trait bound, implies alignment 1
// 3. Composition of `repr(transparent)` `validate_by_slice` with
// `validate_inner` from the `Transparent` trait implies
// valid bytes
// 4. The `repr(transparent)` `validate_byte_slice` implies
// that all bytes are covered
// 5. Composition of `repr(transparent)` `from_byte_slice_unchecked` with
// `cast_ref_unchecked` retains the same reference
// 6. Other methods are left as default
// 7. Equality enforced via `Eq` bound
unsafe impl zerovec::ule::VarULE for $outer
where
$outer: Transparent<$inner> + Eq,
{
#[inline]
fn validate_byte_slice(bytes: &[u8]) -> Result<(), zerovec::ZeroVecError> {
let inner = <$inner>::parse_byte_slice(bytes)?;
Self::validate_inner(inner)
.then_some(())
.ok_or(zerovec::ZeroVecError::parse::<Self>())
}
#[inline]
unsafe fn from_byte_slice_unchecked(bytes: &[u8]) -> &Self {
let inner = <$inner>::from_byte_slice_unchecked(bytes);
Self::cast_ref_unchecked(inner)
}
}
};
}

/// Implements `serde::Deserialize` on a `repr(transparent)` type.
#[allow(unused_macros)]
macro_rules! impl_transparent_serde {
($outer:ident($inner:path)) => {
impl serde::Serialize for $outer {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: serde::ser::Serializer,
Comment on lines +102 to +103
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: This impl is not bound on Transparent because (1) it isn't needed for safety reasons, (2) Transparent is a private trait, and (3) the generated functions use cast_ref_unchecked which in effect enforces transparency since those functions require transparency

{
self.as_inner().serialize(serializer)
}
}
impl<'de, 'a> serde::Deserialize<'de> for &'a $outer
where
'de: 'a,
{
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: serde::Deserializer<'de>,
{
let inner = <&$inner>::deserialize(deserializer)?;
if !$outer::validate_inner(&inner) {
return Err(<D::Error as serde::de::Error>::custom(concat!(
"Failed validation: ",
stringify!($outer)
)));
}
Ok($outer::cast_ref_unchecked(inner))
}
}
impl<'de> serde::Deserialize<'de> for Box<$outer> {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: serde::Deserializer<'de>,
{
let inner = <alloc::boxed::Box<$inner>>::deserialize(deserializer)?;
if !$outer::validate_inner(&inner) {
return Err(<D::Error as serde::de::Error>::custom(concat!(
"Failed validation: ",
stringify!($outer)
)));
}
Ok($outer::cast_box_unchecked(inner))
}
}
};
}
4 changes: 4 additions & 0 deletions components/properties/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,10 @@

extern crate alloc;

#[macro_use]
#[path = "../icu4x_shared.rs"]
mod icu4x_shared;

#[cfg(feature = "bidi")]
pub mod bidi;

Expand Down
50 changes: 22 additions & 28 deletions components/properties/src/provider/names.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@
use alloc::boxed::Box;
use core::cmp::Ordering;

use crate::icu4x_shared::Transparent;
use icu_provider::prelude::*;

use tinystr::TinyStr4;
use zerovec::ule::{UnvalidatedStr, VarULE};
use zerovec::ule::UnvalidatedStr;
use zerovec::{maps::ZeroMapKV, VarZeroSlice, VarZeroVec, ZeroMap, ZeroVec};

/// This is a property name that can be "loose matched" as according to
Expand Down Expand Up @@ -66,35 +66,29 @@ use zerovec::{maps::ZeroMapKV, VarZeroSlice, VarZeroVec, ZeroMap, ZeroVec};
/// assert_eq!(Some(11), map.get_copied_by(|u| u.cmp_loose(key_exact)));
/// ```
#[derive(PartialEq, Eq)] // VarULE wants these to be byte equality
#[derive(Debug, VarULE)]
#[cfg_attr(feature = "serde", derive(serde::Serialize))]
#[derive(Debug)]
#[repr(transparent)]
pub struct NormalizedPropertyNameStr(UnvalidatedStr);

/// This impl requires enabling the optional `serde` Cargo feature of the `icu::properties` crate
#[cfg(feature = "serde")]
impl<'de> serde::Deserialize<'de> for Box<NormalizedPropertyNameStr> {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: serde::Deserializer<'de>,
{
<Box<UnvalidatedStr>>::deserialize(deserializer).map(NormalizedPropertyNameStr::cast_box)
// Safety:
//
// 1. The type is `repr(transparent)` over the inner type
// 2. `validate_inner` is implemented (no invariants)
unsafe impl Transparent<UnvalidatedStr> for NormalizedPropertyNameStr {
#[inline]
fn validate_inner(_inner: &UnvalidatedStr) -> bool {
true
}
#[inline]
fn as_inner(&self) -> &UnvalidatedStr {
&self.0
}
}

/// This impl requires enabling the optional `serde` Cargo feature of the `icu::properties` crate
impl_transparent_helpers!(NormalizedPropertyNameStr(UnvalidatedStr));
impl_transparent_varule!(NormalizedPropertyNameStr(UnvalidatedStr));
#[cfg(feature = "serde")]
impl<'de, 'a> serde::Deserialize<'de> for &'a NormalizedPropertyNameStr
where
'de: 'a,
{
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: serde::Deserializer<'de>,
{
<&UnvalidatedStr>::deserialize(deserializer).map(NormalizedPropertyNameStr::cast_ref)
}
}
impl_transparent_serde!(NormalizedPropertyNameStr(UnvalidatedStr));

impl<'a> ZeroMapKV<'a> for NormalizedPropertyNameStr {
type Container = VarZeroVec<'a, NormalizedPropertyNameStr>;
Expand Down Expand Up @@ -161,14 +155,14 @@ impl NormalizedPropertyNameStr {

/// Convert a [`UnvalidatedStr`] reference to a [`NormalizedPropertyNameStr`] reference.
pub const fn cast_ref(value: &UnvalidatedStr) -> &Self {
// Safety: repr(transparent)
unsafe { core::mem::transmute(value) }
// No invariants: we can directly call cast_*_unchecked
Self::cast_ref_unchecked(value)
}

/// Convert a [`UnvalidatedStr`] box to a [`NormalizedPropertyNameStr`] box.
pub const fn cast_box(value: Box<UnvalidatedStr>) -> Box<Self> {
// Safety: repr(transparent)
unsafe { core::mem::transmute(value) }
// No invariants: we can directly call cast_*_unchecked
Self::cast_box_unchecked(value)
}

/// Get a [`NormalizedPropertyNameStr`] box from a byte slice.
Expand Down
Loading