Cannot get deterministic randomness to work

[JobaDiniz]

models

class Match(BaseModel):
    rule: Rule
    selector: str

class InterestingElement(BaseModel):
    id: str
    html: Html
    match: Match
    selectors: ElementSelector
    text: Text
    nearbyLabels: list["InterestingElement"]

factories

joba = random.Random(10)

@register_fixture
class InterestingElementFactory(ModelFactory[InterestingElement]):
    __set_as_default_factory_for_type__ = True
    __randomize_collection_length__ = True
    __min_collection_length__ = 0
    __max_collection_length__ = 5
    __model__ = InterestingElement
    # __random_seed__ = 10
    __random__ = joba


class RuleFactory(ModelFactory[Rule]):
    __set_as_default_factory_for_type__ = True
    __randomize_collection_length__ = True
    __min_collection_length__ = 0
    __max_collection_length__ = 5
    __model__ = Rule
    __random__ = joba
    # __random_seed__ = 10

    name = Use(ModelFactory.__random__.choice, ["rule", "group_rule", "misc_rule"])
    color = Use(ModelFactory.__random__.choice, ["green", "red", "purple"])
    type = Use(ModelFactory.__random__.choice, ["input", "checkbox", "list", "link"])

The output is always different if I use __random__ instance. If I use __random_seed__ it partially works; with __random_seed__ the name = Use(ModelFactory.__random__.choice, ["rule", "group_rule", "misc_rule"]) doesn't output deterministic data.

[Alc-Alc]

I am assuming this is because of

polyfactory/polyfactory/factories/base.py

Line 504 in d75b1d2

str: cls.__faker__.pystr,

It is directly using the faker instance and the __random__ you set is not being used by faker as it has its own random.Random defined elsewhere (in faker library code).

For a workaround can you try this. I have roughly modeled my own Rule model (based on the overrides in RuleFactory) as you seemed to have missed including that 🙂

from random import Random
from faker import Faker, Generator
from polyfactory.factories.pydantic_factory import ModelFactory
from pydantic import BaseModel

class Rule(BaseModel):
    name: str
    color: str
    type: str
    something: str
    something_else: str

# Create your own faker instance with the random instance you want
joba = Random(10)
generator = Generator()
generator.random = joba
faker = Faker(generator=generator)

class RuleFactory(ModelFactory[Rule]):
    __set_as_default_factory_for_type__ = True
    __randomize_collection_length__ = True
    __min_collection_length__ = 0
    __max_collection_length__ = 5
    # assign the faker and random instances created above
    __random__ = joba
    __faker__ = faker

    # use the `__random__` instance of `RuleFactory` not `ModelFactory`
    name = lambda: RuleFactory.__random__.choice(["rule", "group_rule", "misc_rule"])
    color = lambda: RuleFactory.__random__.choice(["green", "red", "purple"])
    type = lambda: RuleFactory.__random__.choice(["input", "checkbox", "list", "link"])

print(RuleFactory.build())

@guacs can probably add more to this

[guacs]

@JobaDiniz what @Alc-Alc said will work. The reason you were getting non-deterministic values is because 1) faker has it's own seeding and 2) you only set the __random__ on the factories you defined which means ModelFactory.__random__ is the default one set by polyfactory without any specific seed value.

Point 2 is also the reason __random_seed__ didn't work for the Use fields. The easiest way to ensure you don't face any of these issues is to:

from polyfactory.factories.pydantic_factory import ModelFactory

ModelFactory.seed_random(10) # You can run this in an `autouse` fixture in `pytest` as well to get determinstic values across tests

[JobaDiniz]

I liked more this approach, however it isn't working for these fields.

    name = Use(ModelFactory.__random__.choice, ["rule", "group_rule", "misc_rule"])
    color = Use(ModelFactory.__random__.choice, ["green", "red", "purple"])
    type = Use(ModelFactory.__random__.choice, ["input", "checkbox", "list", "link"])

shouldn't ModelFactory.seed_random(10) make the above work?

it only works doing like @Alc-Alc said

    name = lambda: RuleFactory.__random__.choice(["rule", "group_rule", "misc_rule"])
    color = lambda: RuleFactory.__random__.choice(["green", "red", "purple"])
    type = lambda: RuleFactory.__random__.choice(["input", "checkbox", "list", "link"])

But this gives an warning: https://docs.astral.sh/ruff/rules/lambda-assignment/ and it's a little bit ugly.

[Alc-Alc]

I mean you can do this

    @classmethod
    def name(cls) -> str:
        return cls.__random__.choice(["rule", "group_rule", "misc_rule"])
    @classmethod
    def color(cls) -> str:
        return cls.__random__.choice(["green", "red", "purple"])
    @classmethod
    def type(cls) -> str:
        return cls.__random__.choice(["input", "checkbox", "list", "link"])

If this is going into test code, my opinion would be to silence it instead of having to classmethod your way.

[guacs]

I liked more this approach, however it isn't working for these fields.

    name = Use(ModelFactory.__random__.choice, ["rule", "group_rule", "misc_rule"])
    color = Use(ModelFactory.__random__.choice, ["green", "red", "purple"])
    type = Use(ModelFactory.__random__.choice, ["input", "checkbox", "list", "link"])

shouldn't ModelFactory.seed_random(10) make the above work?

it only works doing like @Alc-Alc said

    name = lambda: RuleFactory.__random__.choice(["rule", "group_rule", "misc_rule"])
    color = lambda: RuleFactory.__random__.choice(["green", "red", "purple"])
    type = lambda: RuleFactory.__random__.choice(["input", "checkbox", "list", "link"])

But this gives an warning: https://docs.astral.sh/ruff/rules/lambda-assignment/ and it's a little bit ugly.

Oh thanks for pointing this out! It's due to how we're implementing seed_random. We create a new Random instance with the given seed instead of reseeding the existing instance. This means that the cls.__random__.choice function that's referenced is the method associated with the Random instance before we call seed_random. Thus, you don't get the deterministic behavior.

We should fix this, so that seed_instance reseeds the existing random instance we have instead of creating a new one. As a temporary workaround, you can create your own base ModelFactory with the seed_random method overridden and then use that as the base class for all your factories. Something like the following:

from polyfactory.factories.pydantic_factory import ModelFactory, T

class BaseModelFactory(ModelFactory[T]): # make all your factories inherit from this
    __is_base_factory__ = True

    @classmethod
    def seed_random(cls, seed: int) -> None:
        cls.__random__.seed(seed)
        cls.__faker__.seed_instance(seed)

[guacs]

@JobaDiniz I've created an issue to track this: #580