Module loader (0.3.0)

API documentation for loader module.

Classes

Column

Column(name: str, data_type: str, nullable: bool = True)

Column(name: str, data_type: str, nullable: bool = True)

SpannerDocumentSaver

SpannerDocumentSaver(
    instance_id: str,
    database_id: str,
    table_name: str,
    content_column: str = "page_content",
    metadata_columns: typing.List[str] = [],
    metadata_json_column: str = "langchain_metadata",
    primary_key: typing.Optional[str] = None,
    client: typing.Optional[google.cloud.spanner_v1.client.Client] = None,
)

Save docs to Google Cloud Spanner.

SpannerLoader

SpannerLoader(
    instance_id: str,
    database_id: str,
    query: str,
    content_columns: typing.List[str] = [],
    metadata_columns: typing.List[str] = [],
    format: str = "text",
    databoost: bool = False,
    metadata_json_column: str = "langchain_metadata",
    staleness: typing.Union[float, datetime.datetime] = 0.0,
    client: typing.Optional[google.cloud.spanner_v1.client.Client] = None,
)

Loads data from Google Cloud Spanner.

Modules Functions

_load_doc_to_row

_load_doc_to_row(
    table_fields: typing.List[str],
    doc: langchain_core.documents.base.Document,
    content_column: str,
    metadata_json_column: str,
    parse_json: bool = True,
) -> tuple

Load document to row.

Parameters
Name	Description
`table_fields`	`typing.List[str]` Spanner table fields names.
`doc`	`Document` Document that is used.
`content_column`	`str` Name of the content column.
`metadata_json_column`	`str` Name of the special JSON column.
`parse_json`	`bool` Parse json column to string or leave it as JSON object. String format is needed to for Spanner inserts. JSON object is used to compare with Spanner reads.