Unable to do validation on models.

[jymchng]

Let's supposed I have the following model and create schema.

class ProjectBase(SQLModel):
    name: str = Field(nullable=False)
    repr_name: Optional[str] = Field(default=None, nullable=False)
    is_active: bool = Field(default=True, nullable=False)
    publish_date: date = Field(
        default_factory=datetime.now().date, nullable=False)
    approved: bool = Field(default=False, nullable=False)
    featured: bool = Field(default=False, nullable=False)
    
    # post_init
    @validator("repr_name", always=True)
    def slugify_name(cls, v: Optional[str], values: dict[str, Any]) -> str:
        if v is None:
            logger.info(f"{v=}, {values=}")
            repr_name = slugify(values.get('name'), separator="_")
            return repr_name
        raise ValueError(
            f"`repr_name` cannot be passed in as a parameter to the `Request`.")

As you can see, I am trying to post-process the name field by 'slugify' it.

The ICreateProject schema is as follows:

class IProjectCreate(ProjectBase):
    pass

As I try to create a project using the API, with the following request:

curl -X 'POST' \
  'http://localhost/api/v1/project' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NzY2NTc1MzgsInN1YiI6IjAxODYzZjUwLTkyNWMtNzU3Ny1hYTdmLTc4NmQ1Mjc4M2U2YSIsInR5cGUiOiJhY2Nlc3MifQ.bOtkIl5wJl1ahzGG_A5y7Dei-jxvrmtZalglclXaiR8' \
  -H 'Content-Type: application/json' \
  -d '{
  "is_active": true,
  "name": "Hello A",
  "publish_date": "2023-02-17",
  "approved": false,
  "featured": false
}'

The error log I got is this:

2023-02-18 02:08:37 INFO:     Application startup complete.
2023-02-18 02:08:43 2023-02-17 18:08:43,325 INFO     [project_model.py:29] v=None, values={'name': 'Hello A'}
2023-02-18 02:08:43 2023-02-17 18:08:43,325 INFO     [project_model.py:29] v=None, values={'name': 'Hello A'}
2023-02-18 02:08:43 2023-02-17 18:08:43,326 INFO     [base_crud.py:159] In CRUD.create
2023-02-18 02:08:43 2023-02-17 18:08:43,326 INFO     [base_crud.py:160] obj_in=IProjectCreate(name='Hello A', repr_name='hello_a', is_active=True, publish_date=datetime.date(2023, 2, 17), approved=False, featured=False)
2023-02-18 02:08:43 2023-02-17 18:08:43,326 INFO     [project_model.py:29] v=None, values={}
2023-02-18 02:08:43 INFO:     172.18.0.3:54102 - "POST /api/v1/project HTTP/1.1" 500 Internal Server Error
2023-02-18 02:08:43 ERROR:    Exception in ASGI application
2023-02-18 02:08:43 Traceback (most recent call last):
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/anyio/streams/memory.py", line 94, in receive
2023-02-18 02:08:43     return self.receive_nowait()
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/anyio/streams/memory.py", line 89, in receive_nowait
2023-02-18 02:08:43     raise WouldBlock
2023-02-18 02:08:43 anyio.WouldBlock
2023-02-18 02:08:43 
2023-02-18 02:08:43 During handling of the above exception, another exception occurred:
2023-02-18 02:08:43 
2023-02-18 02:08:43 Traceback (most recent call last):
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/base.py", line 77, in call_next
2023-02-18 02:08:43     message = await recv_stream.receive()
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/anyio/streams/memory.py", line 114, in receive
2023-02-18 02:08:43     raise EndOfStream
2023-02-18 02:08:43 anyio.EndOfStream
2023-02-18 02:08:43 
2023-02-18 02:08:43 During handling of the above exception, another exception occurred:
2023-02-18 02:08:43 
2023-02-18 02:08:43 Traceback (most recent call last):
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 419, in run_asgi
2023-02-18 02:08:43     result = await app(  # type: ignore[func-returns-value]
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
2023-02-18 02:08:43     return await self.app(scope, receive, send)
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/fastapi/applications.py", line 270, in __call__
2023-02-18 02:08:43     await super().__call__(scope, receive, send)
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/starlette/applications.py", line 124, in __call__
2023-02-18 02:08:43     await self.middleware_stack(scope, receive, send)
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
2023-02-18 02:08:43     raise exc
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
2023-02-18 02:08:43     await self.app(scope, receive, _send)
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 92, in __call__
2023-02-18 02:08:43     await self.simple_response(scope, receive, send, request_headers=headers)
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 147, in simple_response
2023-02-18 02:08:43     await self.app(scope, receive, send)
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/base.py", line 106, in __call__
2023-02-18 02:08:43     response = await self.dispatch_func(request, call_next)
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/fastapi_async_sqlalchemy/middleware.py", line 45, in dispatch
2023-02-18 02:08:43     return await call_next(request)
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/base.py", line 80, in call_next
2023-02-18 02:08:43     raise app_exc
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/base.py", line 69, in coro
2023-02-18 02:08:43     await self.app(scope, receive_or_disconnect, send_no_error)
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
2023-02-18 02:08:43     raise exc
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
2023-02-18 02:08:43     await self.app(scope, receive, sender)
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
2023-02-18 02:08:43     raise e
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
2023-02-18 02:08:43     await self.app(scope, receive, send)
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 706, in __call__
2023-02-18 02:08:43     await route.handle(scope, receive, send)
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle
2023-02-18 02:08:43     await self.app(scope, receive, send)
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 66, in app
2023-02-18 02:08:43     response = await func(request)
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 237, in app
2023-02-18 02:08:43     raw_response = await run_endpoint_function(
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 163, in run_endpoint_function
2023-02-18 02:08:43     return await dependant.call(**values)
2023-02-18 02:08:43   File "/code/./app/api/v1/endpoints/project.py", line 83, in create_project
2023-02-18 02:08:43     created = await crud.project.create(obj_in=project, created_by_id=None)
2023-02-18 02:08:43   File "/code/./app/crud/base_crud.py", line 161, in create
2023-02-18 02:08:43     db_obj = self.model.from_orm(obj_in.dict())  # type: ignore
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/sqlmodel/main.py", line 551, in from_orm
2023-02-18 02:08:43     m = cls()
2023-02-18 02:08:43   File "<string>", line 4, in __init__
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/sqlalchemy/orm/state.py", line 481, in _initialize_instance
2023-02-18 02:08:43     with util.safe_reraise():
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
2023-02-18 02:08:43     compat.raise_(
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 208, in raise_
2023-02-18 02:08:43     raise exception
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/sqlalchemy/orm/state.py", line 479, in _initialize_instance
2023-02-18 02:08:43     return manager.original_init(*mixed[1:], **kwargs)
2023-02-18 02:08:43   File "/usr/local/lib/python3.10/site-packages/sqlmodel/main.py", line 498, in __init__
2023-02-18 02:08:43     values, fields_set, validation_error = validate_model(
2023-02-18 02:08:43   File "pydantic/main.py", line 1077, in pydantic.main.validate_model
2023-02-18 02:08:43   File "pydantic/fields.py", line 877, in pydantic.fields.ModelField.validate
2023-02-18 02:08:43   File "pydantic/fields.py", line 1148, in pydantic.fields.ModelField._apply_validators
2023-02-18 02:08:43   File "pydantic/class_validators.py", line 287, in pydantic.class_validators._generic_validator_cls.lambda5
2023-02-18 02:08:43   File "/code/./app/models/project_model.py", line 30, in slugify_name
2023-02-18 02:08:43     repr_name = slugify(values['name'], separator="_")
2023-02-18 02:08:43 KeyError: 'name'

Somehow, the from_orm method does not have the field name.

[jymchng]

I have created a minimum viable codes to illustrate this problem:

from typing import List, Optional, Any
from sqlmodel import Field, SQLModel as _SQLModel, Relationship
from slugify import slugify
from datetime import date, datetime
from pydantic import validator
from sqlalchemy.orm import declared_attr

class SQLModel(_SQLModel):
    
    @declared_attr  # type: ignore
    def __tablename__(cls) -> str:
        return cls.__name__
    
    class Config:
        use_enum_values = True

class BaseIDModel(SQLModel):
    id: Optional[int] = Field(
        primary_key=True,
        index=True,
        nullable=False,
    )

class ProjectBase(SQLModel):
    is_active: bool = Field(default=True, nullable=False)
    name: str = Field(nullable=False)

    publish_date: date = Field(
        default_factory=datetime.now().date, nullable=False)
    repr_name: Optional[str] = Field(default=None, nullable=False)
    
    approved: bool = Field(default=False, nullable=False)
    featured: bool = Field(default=False, nullable=False)  

    # post_init
    @validator("repr_name", pre=True, always=True)
    def slugify_name(cls, v, values) -> str:
        if v is None:
            print(f"{v=}, {values=}")
            return slugify(values["name"], separator="_")
        raise ValueError(
            f"`repr_name` cannot be passed in as a parameter to the `Request`.")
    
class Project(BaseIDModel, ProjectBase, table=True):
    pass

class IProjectCreate(ProjectBase):
    pass

if __name__ == '__main__':
    Project.from_orm(IProjectCreate(name='Hello A'))

The Output is:

v=None, values={'is_active': True, 'name': 'Hello A', 'publish_date': datetime.date(2023, 2, 18)}
v=None, values={'is_active': True, 'publish_date': datetime.date(2023, 2, 18)}
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-4-8a711b7fe15b> in <module>
----> 1 Project.from_orm(IProjectCreate(name='Hello A'))

~\AppData\Roaming\Python\Python38\site-packages\sqlmodel\main.py in from_orm(cls, obj, update)
    549             # If table, create the new instance normally to make SQLAlchemy create
    550             # the _sa_instance_state attribute
--> 551             m = cls()
    552         values, fields_set, validation_error = validate_model(cls, obj)
    553         if validation_error:

<string> in __init__(__pydantic_self__, **data)

~\AppData\Roaming\Python\Python38\site-packages\sqlalchemy\orm\state.py in _initialize_instance(*mixed, **kwargs)
    480         except:
    481             with util.safe_reraise():
--> 482                 manager.dispatch.init_failure(self, args, kwargs)
    483 
    484     def get_history(self, key, passive):

~\AppData\Roaming\Python\Python38\site-packages\sqlalchemy\util\langhelpers.py in __exit__(self, type_, value, traceback)
     68             self._exc_info = None  # remove potential circular references
     69             if not self.warn_only:
---> 70                 compat.raise_(
     71                     exc_value,
     72                     with_traceback=exc_tb,

~\AppData\Roaming\Python\Python38\site-packages\sqlalchemy\util\compat.py in raise_(***failed resolving arguments***)
    206 
    207         try:
--> 208             raise exception
    209         finally:
    210             # credit to

~\AppData\Roaming\Python\Python38\site-packages\sqlalchemy\orm\state.py in _initialize_instance(*mixed, **kwargs)
    477 
    478         try:
--> 479             return manager.original_init(*mixed[1:], **kwargs)
    480         except:
    481             with util.safe_reraise():

~\AppData\Roaming\Python\Python38\site-packages\sqlmodel\main.py in __init__(__pydantic_self__, **data)
    496         # Uses something other than `self` the first arg to allow "self" as a
    497         # settable attribute
--> 498         values, fields_set, validation_error = validate_model(
    499             __pydantic_self__.__class__, data
    500         )

~\AppData\Roaming\Python\Python38\site-packages\pydantic\main.cp38-win_amd64.pyd in pydantic.main.validate_model()

~\AppData\Roaming\Python\Python38\site-packages\pydantic\fields.cp38-win_amd64.pyd in pydantic.fields.ModelField.validate()

~\AppData\Roaming\Python\Python38\site-packages\pydantic\fields.cp38-win_amd64.pyd in pydantic.fields.ModelField._apply_validators()

~\AppData\Roaming\Python\Python38\site-packages\pydantic\class_validators.cp38-win_amd64.pyd in pydantic.class_validators._generic_validator_cls.lambda5()

<ipython-input-2-8fd04dce0748> in slugify_name(cls, v, values)
     31         if v is None:
     32             print(f"{v=}, {values=}")
---> 33             return slugify(values["name"], separator="_")
     34         raise ValueError(
     35             f"`repr_name` cannot be passed in as a parameter to the `Request`.")

KeyError: 'name'

[jymchng]

Opened a discussion in sqlmodel: fastapi/sqlmodel#556

[jonra1993]

Hello @jymchng I see that you got the solution here adding the validation into pydantic schema instead of SQLModel object. I think this validation can be used as a reference and do similar on create.

[jonra1993]

Please check this commit it adds validation on IHereoCreate schema. Also if you ask why schemas use the "I" letter at the beginning this is an internal convention of our team in order to be able to differentiate Models from schemas and as also we use typescript we see schemas similar to interfaces in typescript which we already use "I" convention

[jymchng]

Hi @jonra1993, thank you for working on the commit, yeah, I managed to figure the separation of responsibilities between models and schemas - hence, figured that the right place to do validations is on the schemas.

Thank you for including an example on the IHeroCreate schema for future references.