Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Derive TypeInfo for own types #72

Open
wants to merge 27 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
84b3bb4
Don't use a feature in the derive crate
dvdplm Mar 3, 2021
c2f0f06
Review grumbles
dvdplm Mar 3, 2021
b4ba154
Less noise is better: build a TokenStream2 containing the "::" prefix
dvdplm Mar 3, 2021
b1d26c2
Use `crate::` when deriving for `scale-info` itself
dvdplm Mar 3, 2021
22fb440
Derive TypeInfo for UntrackedSymbol
dvdplm Mar 4, 2021
00aae34
wip
dvdplm Mar 4, 2021
5a165b1
Add ref to ticket that blocks deriving Typeinfo for Type
dvdplm Mar 4, 2021
9ac6306
Remove TypeInfo impl for MetaType
dvdplm Mar 4, 2021
39f8459
Clippy is right
dvdplm Mar 4, 2021
7db6778
Cleanup
dvdplm Mar 4, 2021
687c757
Merge branch 'master' into dp-doogfood
dvdplm Mar 4, 2021
7f4fae7
Merge branch 'master' into dp-doogfood
dvdplm Mar 5, 2021
1d8ff8c
Merge branch 'master' into dp-doogfood
dvdplm Jun 25, 2021
5c22f19
Remove `NonZero*` impls
dvdplm Jun 25, 2021
58dc3d2
Cleanup
dvdplm Jun 25, 2021
e73c061
Don't bind <… as HasCompact>::Type
dvdplm Jun 25, 2021
03bf034
Merge remote-tracking branch 'origin/master' into dp-doogfood
dvdplm Jun 28, 2021
c7bfb52
Review grumbles
dvdplm Jun 28, 2021
89d4499
Fix comment
dvdplm Jun 29, 2021
14055e1
Merge branch 'master' into dp-doogfood
dvdplm Jun 29, 2021
c7c19fc
Address review feedback
dvdplm Jun 29, 2021
d3259c5
fmt
dvdplm Jun 29, 2021
54deaf4
Merge branch 'master' into dp-doogfood
dvdplm Aug 24, 2021
5fd7525
cleanup
dvdplm Aug 24, 2021
2d7530d
Merge branch 'master' into dp-doogfood
gilescope Jul 16, 2022
ba2e94a
Fixup after merge
gilescope Jul 16, 2022
3420903
additional support to allow frame-metadata to derive TypeInfo (#163)
gilescope Jul 19, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,14 @@ std = [
derive = [
"scale-info-derive"
]
# enables decoding and deserialization of portable scale-info type metadata
# Derive `TypeInfo` for our own types.
dogfood = ["derive"]
# Enables decoding and deserialization of portable scale-info type metadata.
dvdplm marked this conversation as resolved.
Show resolved Hide resolved
decode = []

[workspace]
members = [
"derive",
"test_suite",
]
# exclude = ["test_suite"]
69 changes: 51 additions & 18 deletions derive/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.

#![cfg_attr(not(feature = "std"), no_std)]
// #![cfg_attr(not(feature = "std"), no_std)]

extern crate alloc;
extern crate proc_macro;
Expand Down Expand Up @@ -76,7 +76,29 @@ fn generate(input: TokenStream2) -> Result<TokenStream2> {
fn generate_type(input: TokenStream2) -> Result<TokenStream2> {
let mut ast: DeriveInput = syn::parse2(input.clone())?;

let scale_info = crate_name_ident("scale-info")?;
// Find the name given to the `scale-info` crate in the context we run in.
// If scale-info is not among the dependencies then we must be deriving
// types for the scale-info crate itself, in which case we need to rename
// "self" to something, so the object paths keep working.
let (scale_info, import_self_as_scale_info) = {
let actual_crate_name = proc_macro_crate::crate_name("scale-info");
if let Err(e) = actual_crate_name {
if e.starts_with("Could not find `scale-info`") {
(Ident::new("_scale_info", Span::call_site()), true)
dvdplm marked this conversation as resolved.
Show resolved Hide resolved
} else {
return Err(syn::Error::new(Span::call_site(), e))
}
} else {
(
Ident::new(
&actual_crate_name.expect("Checked Err above; qed"),
dvdplm marked this conversation as resolved.
Show resolved Hide resolved
Span::call_site(),
),
false,
)
}
};

let parity_scale_codec = crate_name_ident("parity-scale-codec")?;

let ident = &ast.ident;
Expand All @@ -97,33 +119,44 @@ fn generate_type(input: TokenStream2) -> Result<TokenStream2> {
let generic_type_ids = ast.generics.type_params().map(|ty| {
let ty_ident = &ty.ident;
quote! {
:: #scale_info ::meta_type::<#ty_ident>()
#scale_info::meta_type::<#ty_ident>()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as before for all non-scale-info crates this should still prepend :: to signal root crate path.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't compile with the :: and I don't know why tbh, but I think that when deriving for scale-info itself, and we have extern crate self as _scale_info;, then ::_scale_info::path::to::something doesn't work. :/

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The extern crate self as _scale_info; is not required since Rust edition 2018 and I don't think we should support Rust edition 2015 tbh because it actually resolves many problems we are facing otherwise such as this one.

Copy link
Contributor

@Robbepop Robbepop Mar 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So either this crate should produce: crate:: paths or whatever crate alias a dependency uses for scale-info prepended with ::. To be honest I am not sure how counterintuitive it is to use the derive macros for the crate that exposes them. I see a gain but issues like these let me feel it might be better to simply provide manual implementations for the few types in the re-exporting crate. Happy to learn a better approaches for this problem. In ink! we also use manual implementations for our derive macros in the root crates.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be more concrete: While the proc-macro-crate crate is actually useful in these cases it makes use of file I/O while expanding a proc. macro. There are discussions about encapsulating proc. macros in stricter environments that would break proc. macros such as these. Generally tooling such as rust-analyzer has good reasons why proc. macros that are "impure" should not be relied on.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest I am not sure how counterintuitive it is to use the derive macros for the crate that exposes them.

How do you mean "counterintuitive"? I take it you mean it's forcing things a bit?

I think it's a good litmus test for the crate, especially since the type zoo in scale-info is fairly advanced and gives us a "free test suite" of sorts. The amount of manual implementations we'd have to maintain is pretty large so if we can derive most of them that's a win imo.

I am concerned by the increased complexity this brings. I will see what I can do to make things a wee bit less messy.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am concerned by the increased complexity this brings. I will see what I can do to make things a wee bit less messy.

I think this commit is makes things much cleaner.

Copy link
Contributor

@Robbepop Robbepop Mar 4, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Proc. macros bring overhead with them. So some people prefer using crates but not their provided proc. macros which is why we find derive crate feature on some crates to opt-out. It is just nice to be able to separate concerns between the crate that provides definitions and the crate that provides the proc. macro implementations. As soon as the crate providing definitions makes use of the proc. macro you no longer have the separation and a direct dependency instead. I personally am not against this but overhead can be serious in some cases.
The biggest problem is that derive macros that can be used today in their parent crate require this type of "hack" that is encapsulated by the proc-macro-crate crate currently. Using this "hack" in any proc. macro turns the proc. macro impure. For example it can no longer enjoy acceleration by the techniques behind the watt crate which some people believe to be the future compilation model for all proc. macros since it is super fast and more secure than today's proc. macros.

Don't get me wrong: I see the tradeoffs and see some of the pros of doing this. But I still consider this feature to be taking in some amount of technical depth into the project in order to have the anticipated feature implementation for which we will probably have to pay back in the future. Technical depth is death to all progress so I always am careful how much of it I accumulate in projects.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. The use of proc-macro-crate predates this PR so the technical debt is already there;
  2. This PR does not introduce a mandatory dependence on scale-info-derive: it's opt-in through a feature (as it should be, couldn't agree with you more);

I think we should have a conversation about the use of proc-macro-crate; you make a really good point about it making the crate "impure" and we should hash out the trade-offs involved. But I'm not sure this PR is the right place for that?

}
});

let ast: DeriveInput = syn::parse2(input.clone())?;
let mut ast: DeriveInput = syn::parse2(input.clone())?;
let build_type = match &ast.data {
Data::Struct(ref s) => generate_composite_type(s, &scale_info),
Data::Enum(ref e) => generate_variant_type(e, &scale_info),
Data::Union(_) => return Err(Error::new_spanned(input, "Unions not supported")),
};

// Remove any type parameter defaults, we don't want those. E.g. `impl<T: Stuff = WutEven>`
ast.generics.type_params_mut().for_each(|type_param| {
type_param.default = None;
});
let generic_types = ast.generics.type_params();
let type_info_impl = quote! {
impl <#( #generic_types ),*> :: #scale_info ::TypeInfo for #ident #ty_generics #where_clause {
impl <#( #generic_types ),*> #scale_info::TypeInfo for #ident #ty_generics #where_clause {
type Identity = Self;
fn type_info() -> :: #scale_info ::Type {
:: #scale_info ::Type::builder()
.path(:: #scale_info ::Path::new(stringify!(#ident), module_path!()))
.type_params(:: #scale_info ::prelude::vec![ #( #generic_type_ids ),* ])
fn type_info() -> #scale_info::Type {
#scale_info::Type::builder()
.path(#scale_info::Path::new(stringify!(#ident), module_path!()))
.type_params(#scale_info::prelude::vec![ #( #generic_type_ids ),* ])
.#build_type
}
}
};
};
let crate_rename = if import_self_as_scale_info {
quote! { extern crate self as _scale_info; }
} else {
quote! {}
};

Ok(quote! {
#[allow(non_upper_case_globals, unused_attributes, unused_qualifications)]
// TODO: are these needed?
// #[allow(non_upper_case_globals, unused_attributes, unused_qualifications)]
const _: () = {
#type_info_impl;
#crate_rename
#type_info_impl
};
})
}
Expand Down Expand Up @@ -222,7 +255,7 @@ fn generate_composite_type(data_struct: &DataStruct, scale_info: &Ident) -> Toke
}
};
quote! {
composite(:: #scale_info ::build::Fields::#fields)
composite(#scale_info::build::Fields::#fields)
}
}

Expand Down Expand Up @@ -252,7 +285,7 @@ fn generate_c_like_enum_def(variants: &VariantList, scale_info: &Ident) -> Token
});
quote! {
variant(
:: #scale_info ::build::Variants::fieldless()
#scale_info::build::Variants::fieldless()
#( #variants )*
)
}
Expand Down Expand Up @@ -281,7 +314,7 @@ fn generate_variant_type(data_enum: &DataEnum, scale_info: &Ident) -> TokenStrea
quote! {
.variant(
#v_name,
:: #scale_info ::build::Fields::named()
#scale_info::build::Fields::named()
#( #fields)*
)
}
Expand All @@ -291,7 +324,7 @@ fn generate_variant_type(data_enum: &DataEnum, scale_info: &Ident) -> TokenStrea
quote! {
.variant(
#v_name,
:: #scale_info ::build::Fields::unnamed()
#scale_info::build::Fields::unnamed()
#( #fields)*
)
}
Expand All @@ -305,7 +338,7 @@ fn generate_variant_type(data_enum: &DataEnum, scale_info: &Ident) -> TokenStrea
});
quote! {
variant(
:: #scale_info ::build::Variants::with_fields()
#scale_info::build::Variants::with_fields()
#( #variants)*
)
}
Expand Down
4 changes: 2 additions & 2 deletions derive/src/trait_bounds.rs
Original file line number Diff line number Diff line change
Expand Up @@ -66,14 +66,14 @@ pub fn make_where_clause<'a>(
} else {
where_clause
.predicates
.push(parse_quote!(#ty : :: #scale_info ::TypeInfo + 'static));
.push(parse_quote!(#ty : #scale_info ::TypeInfo + 'static));
}
});

generics.type_params().into_iter().for_each(|type_param| {
let ident = type_param.ident.clone();
let mut bounds = type_param.bounds.clone();
bounds.push(parse_quote!(:: #scale_info ::TypeInfo));
bounds.push(parse_quote!(#scale_info ::TypeInfo));
bounds.push(parse_quote!('static));
where_clause
.predicates
Expand Down
1 change: 1 addition & 0 deletions src/form.rs
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ pub trait Form {
/// through the registry and `IntoPortable`.
#[cfg_attr(feature = "serde", derive(Serialize))]
#[derive(PartialEq, Eq, PartialOrd, Ord, Clone, Copy, Debug)]
#[cfg_attr(feature = "dogfood", derive(scale_info_derive::TypeInfo))]
pub enum MetaForm {}

impl Form for MetaForm {
Expand Down
40 changes: 40 additions & 0 deletions src/impls.rs
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,18 @@ use crate::{
TypeDefTuple,
TypeInfo,
};
use core::num::{
NonZeroI128,
NonZeroI16,
NonZeroI32,
NonZeroI64,
NonZeroI8,
NonZeroU128,
NonZeroU16,
NonZeroU32,
NonZeroU64,
NonZeroU8,
};

macro_rules! impl_metadata_for_primitives {
( $( $t:ty => $ident_kind:expr, )* ) => { $(
Expand Down Expand Up @@ -119,6 +131,34 @@ impl_metadata_for_tuple!(A, B, C, D, E, F, G, H, I, J, K, L, M, N);
impl_metadata_for_tuple!(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O);
impl_metadata_for_tuple!(A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P);

macro_rules! impl_for_non_zero {
( $( $t: ty ),* $(,)? ) => {
$(
impl TypeInfo for $t {
type Identity = Self;
fn type_info() -> Type {
Type::builder()
.path(Path::prelude(stringify!($t)))
.composite(Fields::unnamed().field_of::<$t>(stringify!($t)))
}
}
)*
};
}

impl_for_non_zero!(
NonZeroI8,
NonZeroI16,
NonZeroI32,
NonZeroI64,
NonZeroI128,
NonZeroU8,
NonZeroU16,
NonZeroU32,
NonZeroU64,
NonZeroU128
);

impl<T> TypeInfo for Vec<T>
where
T: TypeInfo + 'static,
Expand Down
21 changes: 21 additions & 0 deletions src/interner.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,14 @@ use crate::prelude::{
vec::Vec,
};

use crate::{
build::Fields,
form::MetaForm,
Path,
Type,
TypeInfo,
};

#[cfg(feature = "serde")]
use serde::{
Deserialize,
Expand Down Expand Up @@ -72,6 +80,19 @@ impl<T> scale::Decode for UntrackedSymbol<T> {
}
}

// TODO: we should be able to derive this.
impl<T> TypeInfo for UntrackedSymbol<T>
where
T: TypeInfo + 'static,
{
type Identity = Self;
fn type_info() -> Type<MetaForm> {
Type::builder()
.path(Path::prelude("Path"))
.composite(Fields::named().field_of::<NonZeroU32>("id", "NonZeroU32"))
}
}

impl<T> UntrackedSymbol<T> {
/// Returns the index to the symbol in the interner table.
pub fn id(&self) -> NonZeroU32 {
Expand Down
14 changes: 14 additions & 0 deletions src/meta_type.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,9 @@ use crate::prelude::{
};

use crate::{
build::Fields,
form::MetaForm,
Path,
Type,
TypeInfo,
};
Expand All @@ -49,6 +51,18 @@ pub struct MetaType {
type_id: TypeId,
}

// TODO: this is a total hack. Not sure what we can do here.
impl TypeInfo for MetaType {
type Identity = Self;
fn type_info() -> Type<MetaForm> {
Type::builder()
.path(Path::new("MetaType", "meta_type"))
.composite(
Fields::named().field_of::<core::num::NonZeroU64>("type_id", "TypeId"),
)
}
}

impl PartialEq for MetaType {
fn eq(&self, other: &Self) -> bool {
self.type_id == other.type_id
Expand Down
10 changes: 10 additions & 0 deletions src/registry.rs
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ use crate::{
use scale::Encode;
#[cfg(feature = "serde")]
use serde::{
de::DeserializeOwned,
Deserialize,
Serialize,
};
Expand Down Expand Up @@ -166,6 +167,15 @@ impl Registry {

/// A read-only registry containing types in their portable form for serialization.
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
// #[cfg_attr(feature = "serde", derive(Serialize))]
// serialize = "T::Type: Serialize, T::String: Serialize",
// TODO: do we need this?
#[cfg_attr(
feature = "serde",
serde(bound(
deserialize = "<PortableForm as Form>::Type: DeserializeOwned, <PortableForm as Form>::String: DeserializeOwned",
))
)]
#[cfg_attr(any(feature = "std", feature = "decode"), derive(scale::Decode))]
#[derive(Clone, Debug, PartialEq, Eq, Encode)]
pub struct PortableRegistry {
Expand Down
1 change: 1 addition & 0 deletions src/ty/composite.rs
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ use serde::{
#[cfg_attr(feature = "serde", serde(rename_all = "lowercase"))]
#[cfg_attr(any(feature = "std", feature = "decode"), derive(scale::Decode))]
#[derive(PartialEq, Eq, PartialOrd, Ord, Clone, Debug, From, Encode)]
#[cfg_attr(feature = "dogfood", derive(scale_info_derive::TypeInfo))]
pub struct TypeDefComposite<T: Form = MetaForm> {
/// The fields of the composite type.
#[cfg_attr(
Expand Down
1 change: 1 addition & 0 deletions src/ty/fields.rs
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ use serde::{
#[cfg_attr(feature = "serde", serde(rename_all = "camelCase"))]
#[cfg_attr(any(feature = "std", feature = "decode"), derive(scale::Decode))]
#[derive(PartialEq, Eq, PartialOrd, Ord, Clone, Debug, Encode)]
#[cfg_attr(feature = "dogfood", derive(scale_info_derive::TypeInfo))]
pub struct Field<T: Form = MetaForm> {
/// The name of the field. None for unnamed fields.
#[cfg_attr(
Expand Down
Loading