# Data Sources ## List Data Sources `List dataSources().list(DataSourceListParamsparams = DataSourceListParams.none(), RequestOptionsrequestOptions = RequestOptions.none())` **get** `/api/v1/data-sources` List data sources for a given project. If project_id is not provided, uses the default project. ### Parameters - `DataSourceListParams params` - `Optional organizationId` - `Optional projectId` ### Example ```java package com.llamacloud_prod.api.example; import com.llamacloud_prod.api.client.LlamaCloudClient; import com.llamacloud_prod.api.client.okhttp.LlamaCloudOkHttpClient; import com.llamacloud_prod.api.models.datasources.DataSource; import com.llamacloud_prod.api.models.datasources.DataSourceListParams; public final class Main { private Main() {} public static void main(String[] args) { LlamaCloudClient client = LlamaCloudOkHttpClient.fromEnv(); List dataSources = client.dataSources().list(); } } ``` #### Response ```json [ { "id": "182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e", "component": { "foo": "bar" }, "name": "name", "project_id": "182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e", "source_type": "S3", "created_at": "2019-12-27T18:11:19.117Z", "custom_metadata": { "foo": { "foo": "bar" } }, "updated_at": "2019-12-27T18:11:19.117Z", "version_metadata": { "reader_version": "1.0" } } ] ``` ## Create Data Source `DataSource dataSources().create(DataSourceCreateParamsparams, RequestOptionsrequestOptions = RequestOptions.none())` **post** `/api/v1/data-sources` Create a new data source. ### Parameters - `DataSourceCreateParams params` - `Optional organizationId` - `Optional projectId` - `Component component` Component that implements the data source - `class UnionMember0:` - `class CloudS3DataSource:` - `String bucket` The name of the S3 bucket to read from. - `Optional awsAccessId` The AWS access ID to use for authentication. - `Optional awsAccessSecret` The AWS access secret to use for authentication. - `Optional className` - `Optional prefix` The prefix of the S3 objects to read from. - `Optional regexPattern` The regex pattern to filter S3 objects. Must be a valid regex pattern. - `Optional s3EndpointUrl` The S3 endpoint URL to use for authentication. - `Optional supportsAccessControl` - `class CloudAzStorageBlobDataSource:` - `String accountUrl` The Azure Storage Blob account URL to use for authentication. - `String containerName` The name of the Azure Storage Blob container to read from. - `Optional accountKey` The Azure Storage Blob account key to use for authentication. - `Optional accountName` The Azure Storage Blob account name to use for authentication. - `Optional blob` The blob name to read from. - `Optional className` - `Optional clientId` The Azure AD client ID to use for authentication. - `Optional clientSecret` The Azure AD client secret to use for authentication. - `Optional prefix` The prefix of the Azure Storage Blob objects to read from. - `Optional supportsAccessControl` - `Optional tenantId` The Azure AD tenant ID to use for authentication. - `class CloudGoogleDriveDataSource:` - `String folderId` The ID of the Google Drive folder to read from. - `Optional className` - `Optional serviceAccountKey` A dictionary containing secret values - `Optional supportsAccessControl` - `class CloudOneDriveDataSource:` - `String clientId` The client ID to use for authentication. - `String clientSecret` The client secret to use for authentication. - `String tenantId` The tenant ID to use for authentication. - `String userPrincipalName` The user principal name to use for authentication. - `Optional className` - `Optional folderId` The ID of the OneDrive folder to read from. - `Optional folderPath` The path of the OneDrive folder to read from. - `Optional> requiredExts` The list of required file extensions. - `Optional supportsAccessControl` - `TRUE(true)` - `class CloudSharepointDataSource:` - `String clientId` The client ID to use for authentication. - `String clientSecret` The client secret to use for authentication. - `String tenantId` The tenant ID to use for authentication. - `Optional className` - `Optional driveName` The name of the Sharepoint drive to read from. - `Optional> excludePathPatterns` List of regex patterns for file paths to exclude. Files whose paths (including filename) match any pattern will be excluded. Example: ['/temp/', '/backup/', '.git/', '.tmp$', '^~'] - `Optional folderId` The ID of the Sharepoint folder to read from. - `Optional folderPath` The path of the Sharepoint folder to read from. - `Optional getPermissions` Whether to get permissions for the sharepoint site. - `Optional> includePathPatterns` List of regex patterns for file paths to include. Full paths (including filename) must match at least one pattern to be included. Example: ['/reports/', '/docs/.*.pdf$', '^Report.*.pdf$'] - `Optional> requiredExts` The list of required file extensions. - `Optional siteId` The ID of the SharePoint site to download from. - `Optional siteName` The name of the SharePoint site to download from. - `Optional supportsAccessControl` - `TRUE(true)` - `class CloudSlackDataSource:` - `String slackToken` Slack Bot Token. - `Optional channelIds` Slack Channel. - `Optional channelPatterns` Slack Channel name pattern. - `Optional className` - `Optional earliestDate` Earliest date. - `Optional earliestDateTimestamp` Earliest date timestamp. - `Optional latestDate` Latest date. - `Optional latestDateTimestamp` Latest date timestamp. - `Optional supportsAccessControl` - `class CloudNotionPageDataSource:` - `String integrationToken` The integration token to use for authentication. - `Optional className` - `Optional databaseIds` The Notion Database Id to read content from. - `Optional pageIds` The Page ID's of the Notion to read from. - `Optional supportsAccessControl` - `class CloudConfluenceDataSource:` - `String authenticationMechanism` Type of Authentication for connecting to Confluence APIs. - `String serverUrl` The server URL of the Confluence instance. - `Optional apiToken` The API token to use for authentication. - `Optional className` - `Optional cql` The CQL query to use for fetching pages. - `Optional failureHandling` Configuration for handling failures during processing. Key-value object controlling failure handling behaviors. Example: { "skip_list_failures": true } Currently supports: - skip_list_failures: Skip failed batches/lists and continue processing - `Optional skipListFailures` Whether to skip failed batches/lists and continue processing - `Optional indexRestrictedPages` Whether to index restricted pages. - `Optional keepMarkdownFormat` Whether to keep the markdown format. - `Optional label` The label to use for fetching pages. - `Optional pageIds` The page IDs of the Confluence to read from. - `Optional spaceKey` The space key to read from. - `Optional supportsAccessControl` - `Optional userName` The username to use for authentication. - `class CloudJiraDataSource:` Cloud Jira Data Source integrating JiraReader. - `String authenticationMechanism` Type of Authentication for connecting to Jira APIs. - `String query` JQL (Jira Query Language) query to search. - `Optional apiToken` The API/ Access Token used for Basic, PAT and OAuth2 authentication. - `Optional className` - `Optional cloudId` The cloud ID, used in case of OAuth2. - `Optional email` The email address to use for authentication. - `Optional serverUrl` The server url for Jira Cloud. - `Optional supportsAccessControl` - `class CloudJiraDataSourceV2:` Cloud Jira Data Source integrating JiraReaderV2. - `String authenticationMechanism` Type of Authentication for connecting to Jira APIs. - `String query` JQL (Jira Query Language) query to search. - `String serverUrl` The server url for Jira Cloud. - `Optional apiToken` The API Access Token used for Basic, PAT and OAuth2 authentication. - `Optional apiVersion` Jira REST API version to use (2 or 3). 3 supports Atlassian Document Format (ADF). - `_2("2")` - `_3("3")` - `Optional className` - `Optional cloudId` The cloud ID, used in case of OAuth2. - `Optional email` The email address to use for authentication. - `Optional expand` Fields to expand in the response. - `Optional> fields` List of fields to retrieve from Jira. If None, retrieves all fields. - `Optional getPermissions` Whether to fetch project role permissions and issue-level security - `Optional requestsPerMinute` Rate limit for Jira API requests per minute. - `Optional supportsAccessControl` - `class CloudBoxDataSource:` - `AuthenticationMechanism authenticationMechanism` The type of authentication to use (Developer Token or CCG) - `DEVELOPER_TOKEN("developer_token")` - `CCG("ccg")` - `Optional className` - `Optional clientId` Box API key used for identifying the application the user is authenticating with - `Optional clientSecret` Box API secret used for making auth requests. - `Optional developerToken` Developer token for authentication if authentication_mechanism is 'developer_token'. - `Optional enterpriseId` Box Enterprise ID, if provided authenticates as service. - `Optional folderId` The ID of the Box folder to read from. - `Optional supportsAccessControl` - `Optional userId` Box User ID, if provided authenticates as user. - `String name` The name of the data source. - `SourceType sourceType` - `S3("S3")` - `AZURE_STORAGE_BLOB("AZURE_STORAGE_BLOB")` - `GOOGLE_DRIVE("GOOGLE_DRIVE")` - `MICROSOFT_ONEDRIVE("MICROSOFT_ONEDRIVE")` - `MICROSOFT_SHAREPOINT("MICROSOFT_SHAREPOINT")` - `SLACK("SLACK")` - `NOTION_PAGE("NOTION_PAGE")` - `CONFLUENCE("CONFLUENCE")` - `JIRA("JIRA")` - `JIRA_V2("JIRA_V2")` - `BOX("BOX")` - `Optional customMetadata` Custom metadata that will be present on all data loaded from the data source - `class UnionMember0:` - `List` - `String` - `double` - `boolean` ### Returns - `class DataSource:` Schema for a data source. - `String id` Unique identifier - `Component component` Component that implements the data source - `class UnionMember0:` - `class CloudS3DataSource:` - `String bucket` The name of the S3 bucket to read from. - `Optional awsAccessId` The AWS access ID to use for authentication. - `Optional awsAccessSecret` The AWS access secret to use for authentication. - `Optional className` - `Optional prefix` The prefix of the S3 objects to read from. - `Optional regexPattern` The regex pattern to filter S3 objects. Must be a valid regex pattern. - `Optional s3EndpointUrl` The S3 endpoint URL to use for authentication. - `Optional supportsAccessControl` - `class CloudAzStorageBlobDataSource:` - `String accountUrl` The Azure Storage Blob account URL to use for authentication. - `String containerName` The name of the Azure Storage Blob container to read from. - `Optional accountKey` The Azure Storage Blob account key to use for authentication. - `Optional accountName` The Azure Storage Blob account name to use for authentication. - `Optional blob` The blob name to read from. - `Optional className` - `Optional clientId` The Azure AD client ID to use for authentication. - `Optional clientSecret` The Azure AD client secret to use for authentication. - `Optional prefix` The prefix of the Azure Storage Blob objects to read from. - `Optional supportsAccessControl` - `Optional tenantId` The Azure AD tenant ID to use for authentication. - `class CloudGoogleDriveDataSource:` - `String folderId` The ID of the Google Drive folder to read from. - `Optional className` - `Optional serviceAccountKey` A dictionary containing secret values - `Optional supportsAccessControl` - `class CloudOneDriveDataSource:` - `String clientId` The client ID to use for authentication. - `String clientSecret` The client secret to use for authentication. - `String tenantId` The tenant ID to use for authentication. - `String userPrincipalName` The user principal name to use for authentication. - `Optional className` - `Optional folderId` The ID of the OneDrive folder to read from. - `Optional folderPath` The path of the OneDrive folder to read from. - `Optional> requiredExts` The list of required file extensions. - `Optional supportsAccessControl` - `TRUE(true)` - `class CloudSharepointDataSource:` - `String clientId` The client ID to use for authentication. - `String clientSecret` The client secret to use for authentication. - `String tenantId` The tenant ID to use for authentication. - `Optional className` - `Optional driveName` The name of the Sharepoint drive to read from. - `Optional> excludePathPatterns` List of regex patterns for file paths to exclude. Files whose paths (including filename) match any pattern will be excluded. Example: ['/temp/', '/backup/', '.git/', '.tmp$', '^~'] - `Optional folderId` The ID of the Sharepoint folder to read from. - `Optional folderPath` The path of the Sharepoint folder to read from. - `Optional getPermissions` Whether to get permissions for the sharepoint site. - `Optional> includePathPatterns` List of regex patterns for file paths to include. Full paths (including filename) must match at least one pattern to be included. Example: ['/reports/', '/docs/.*.pdf$', '^Report.*.pdf$'] - `Optional> requiredExts` The list of required file extensions. - `Optional siteId` The ID of the SharePoint site to download from. - `Optional siteName` The name of the SharePoint site to download from. - `Optional supportsAccessControl` - `TRUE(true)` - `class CloudSlackDataSource:` - `String slackToken` Slack Bot Token. - `Optional channelIds` Slack Channel. - `Optional channelPatterns` Slack Channel name pattern. - `Optional className` - `Optional earliestDate` Earliest date. - `Optional earliestDateTimestamp` Earliest date timestamp. - `Optional latestDate` Latest date. - `Optional latestDateTimestamp` Latest date timestamp. - `Optional supportsAccessControl` - `class CloudNotionPageDataSource:` - `String integrationToken` The integration token to use for authentication. - `Optional className` - `Optional databaseIds` The Notion Database Id to read content from. - `Optional pageIds` The Page ID's of the Notion to read from. - `Optional supportsAccessControl` - `class CloudConfluenceDataSource:` - `String authenticationMechanism` Type of Authentication for connecting to Confluence APIs. - `String serverUrl` The server URL of the Confluence instance. - `Optional apiToken` The API token to use for authentication. - `Optional className` - `Optional cql` The CQL query to use for fetching pages. - `Optional failureHandling` Configuration for handling failures during processing. Key-value object controlling failure handling behaviors. Example: { "skip_list_failures": true } Currently supports: - skip_list_failures: Skip failed batches/lists and continue processing - `Optional skipListFailures` Whether to skip failed batches/lists and continue processing - `Optional indexRestrictedPages` Whether to index restricted pages. - `Optional keepMarkdownFormat` Whether to keep the markdown format. - `Optional label` The label to use for fetching pages. - `Optional pageIds` The page IDs of the Confluence to read from. - `Optional spaceKey` The space key to read from. - `Optional supportsAccessControl` - `Optional userName` The username to use for authentication. - `class CloudJiraDataSource:` Cloud Jira Data Source integrating JiraReader. - `String authenticationMechanism` Type of Authentication for connecting to Jira APIs. - `String query` JQL (Jira Query Language) query to search. - `Optional apiToken` The API/ Access Token used for Basic, PAT and OAuth2 authentication. - `Optional className` - `Optional cloudId` The cloud ID, used in case of OAuth2. - `Optional email` The email address to use for authentication. - `Optional serverUrl` The server url for Jira Cloud. - `Optional supportsAccessControl` - `class CloudJiraDataSourceV2:` Cloud Jira Data Source integrating JiraReaderV2. - `String authenticationMechanism` Type of Authentication for connecting to Jira APIs. - `String query` JQL (Jira Query Language) query to search. - `String serverUrl` The server url for Jira Cloud. - `Optional apiToken` The API Access Token used for Basic, PAT and OAuth2 authentication. - `Optional apiVersion` Jira REST API version to use (2 or 3). 3 supports Atlassian Document Format (ADF). - `_2("2")` - `_3("3")` - `Optional className` - `Optional cloudId` The cloud ID, used in case of OAuth2. - `Optional email` The email address to use for authentication. - `Optional expand` Fields to expand in the response. - `Optional> fields` List of fields to retrieve from Jira. If None, retrieves all fields. - `Optional getPermissions` Whether to fetch project role permissions and issue-level security - `Optional requestsPerMinute` Rate limit for Jira API requests per minute. - `Optional supportsAccessControl` - `class CloudBoxDataSource:` - `AuthenticationMechanism authenticationMechanism` The type of authentication to use (Developer Token or CCG) - `DEVELOPER_TOKEN("developer_token")` - `CCG("ccg")` - `Optional className` - `Optional clientId` Box API key used for identifying the application the user is authenticating with - `Optional clientSecret` Box API secret used for making auth requests. - `Optional developerToken` Developer token for authentication if authentication_mechanism is 'developer_token'. - `Optional enterpriseId` Box Enterprise ID, if provided authenticates as service. - `Optional folderId` The ID of the Box folder to read from. - `Optional supportsAccessControl` - `Optional userId` Box User ID, if provided authenticates as user. - `String name` The name of the data source. - `String projectId` - `SourceType sourceType` - `S3("S3")` - `AZURE_STORAGE_BLOB("AZURE_STORAGE_BLOB")` - `GOOGLE_DRIVE("GOOGLE_DRIVE")` - `MICROSOFT_ONEDRIVE("MICROSOFT_ONEDRIVE")` - `MICROSOFT_SHAREPOINT("MICROSOFT_SHAREPOINT")` - `SLACK("SLACK")` - `NOTION_PAGE("NOTION_PAGE")` - `CONFLUENCE("CONFLUENCE")` - `JIRA("JIRA")` - `JIRA_V2("JIRA_V2")` - `BOX("BOX")` - `Optional createdAt` Creation datetime - `Optional customMetadata` Custom metadata that will be present on all data loaded from the data source - `class UnionMember0:` - `List` - `String` - `double` - `boolean` - `Optional updatedAt` Update datetime - `Optional versionMetadata` Version metadata for the data source - `Optional readerVersion` The version of the reader to use for this data source. - `_1_0("1.0")` - `_2_0("2.0")` - `_2_1("2.1")` ### Example ```java package com.llamacloud_prod.api.example; import com.llamacloud_prod.api.client.LlamaCloudClient; import com.llamacloud_prod.api.client.okhttp.LlamaCloudOkHttpClient; import com.llamacloud_prod.api.core.JsonValue; import com.llamacloud_prod.api.models.datasources.DataSource; import com.llamacloud_prod.api.models.datasources.DataSourceCreateParams; public final class Main { private Main() {} public static void main(String[] args) { LlamaCloudClient client = LlamaCloudOkHttpClient.fromEnv(); DataSourceCreateParams params = DataSourceCreateParams.builder() .component(DataSourceCreateParams.Component.UnionMember0.builder() .putAdditionalProperty("foo", JsonValue.from("bar")) .build()) .name("name") .sourceType(DataSourceCreateParams.SourceType.S3) .build(); DataSource dataSource = client.dataSources().create(params); } } ``` #### Response ```json { "id": "182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e", "component": { "foo": "bar" }, "name": "name", "project_id": "182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e", "source_type": "S3", "created_at": "2019-12-27T18:11:19.117Z", "custom_metadata": { "foo": { "foo": "bar" } }, "updated_at": "2019-12-27T18:11:19.117Z", "version_metadata": { "reader_version": "1.0" } } ``` ## Get Data Source `DataSource dataSources().get(DataSourceGetParamsparams = DataSourceGetParams.none(), RequestOptionsrequestOptions = RequestOptions.none())` **get** `/api/v1/data-sources/{data_source_id}` Get a data source by ID. ### Parameters - `DataSourceGetParams params` - `Optional dataSourceId` ### Returns - `class DataSource:` Schema for a data source. - `String id` Unique identifier - `Component component` Component that implements the data source - `class UnionMember0:` - `class CloudS3DataSource:` - `String bucket` The name of the S3 bucket to read from. - `Optional awsAccessId` The AWS access ID to use for authentication. - `Optional awsAccessSecret` The AWS access secret to use for authentication. - `Optional className` - `Optional prefix` The prefix of the S3 objects to read from. - `Optional regexPattern` The regex pattern to filter S3 objects. Must be a valid regex pattern. - `Optional s3EndpointUrl` The S3 endpoint URL to use for authentication. - `Optional supportsAccessControl` - `class CloudAzStorageBlobDataSource:` - `String accountUrl` The Azure Storage Blob account URL to use for authentication. - `String containerName` The name of the Azure Storage Blob container to read from. - `Optional accountKey` The Azure Storage Blob account key to use for authentication. - `Optional accountName` The Azure Storage Blob account name to use for authentication. - `Optional blob` The blob name to read from. - `Optional className` - `Optional clientId` The Azure AD client ID to use for authentication. - `Optional clientSecret` The Azure AD client secret to use for authentication. - `Optional prefix` The prefix of the Azure Storage Blob objects to read from. - `Optional supportsAccessControl` - `Optional tenantId` The Azure AD tenant ID to use for authentication. - `class CloudGoogleDriveDataSource:` - `String folderId` The ID of the Google Drive folder to read from. - `Optional className` - `Optional serviceAccountKey` A dictionary containing secret values - `Optional supportsAccessControl` - `class CloudOneDriveDataSource:` - `String clientId` The client ID to use for authentication. - `String clientSecret` The client secret to use for authentication. - `String tenantId` The tenant ID to use for authentication. - `String userPrincipalName` The user principal name to use for authentication. - `Optional className` - `Optional folderId` The ID of the OneDrive folder to read from. - `Optional folderPath` The path of the OneDrive folder to read from. - `Optional> requiredExts` The list of required file extensions. - `Optional supportsAccessControl` - `TRUE(true)` - `class CloudSharepointDataSource:` - `String clientId` The client ID to use for authentication. - `String clientSecret` The client secret to use for authentication. - `String tenantId` The tenant ID to use for authentication. - `Optional className` - `Optional driveName` The name of the Sharepoint drive to read from. - `Optional> excludePathPatterns` List of regex patterns for file paths to exclude. Files whose paths (including filename) match any pattern will be excluded. Example: ['/temp/', '/backup/', '.git/', '.tmp$', '^~'] - `Optional folderId` The ID of the Sharepoint folder to read from. - `Optional folderPath` The path of the Sharepoint folder to read from. - `Optional getPermissions` Whether to get permissions for the sharepoint site. - `Optional> includePathPatterns` List of regex patterns for file paths to include. Full paths (including filename) must match at least one pattern to be included. Example: ['/reports/', '/docs/.*.pdf$', '^Report.*.pdf$'] - `Optional> requiredExts` The list of required file extensions. - `Optional siteId` The ID of the SharePoint site to download from. - `Optional siteName` The name of the SharePoint site to download from. - `Optional supportsAccessControl` - `TRUE(true)` - `class CloudSlackDataSource:` - `String slackToken` Slack Bot Token. - `Optional channelIds` Slack Channel. - `Optional channelPatterns` Slack Channel name pattern. - `Optional className` - `Optional earliestDate` Earliest date. - `Optional earliestDateTimestamp` Earliest date timestamp. - `Optional latestDate` Latest date. - `Optional latestDateTimestamp` Latest date timestamp. - `Optional supportsAccessControl` - `class CloudNotionPageDataSource:` - `String integrationToken` The integration token to use for authentication. - `Optional className` - `Optional databaseIds` The Notion Database Id to read content from. - `Optional pageIds` The Page ID's of the Notion to read from. - `Optional supportsAccessControl` - `class CloudConfluenceDataSource:` - `String authenticationMechanism` Type of Authentication for connecting to Confluence APIs. - `String serverUrl` The server URL of the Confluence instance. - `Optional apiToken` The API token to use for authentication. - `Optional className` - `Optional cql` The CQL query to use for fetching pages. - `Optional failureHandling` Configuration for handling failures during processing. Key-value object controlling failure handling behaviors. Example: { "skip_list_failures": true } Currently supports: - skip_list_failures: Skip failed batches/lists and continue processing - `Optional skipListFailures` Whether to skip failed batches/lists and continue processing - `Optional indexRestrictedPages` Whether to index restricted pages. - `Optional keepMarkdownFormat` Whether to keep the markdown format. - `Optional label` The label to use for fetching pages. - `Optional pageIds` The page IDs of the Confluence to read from. - `Optional spaceKey` The space key to read from. - `Optional supportsAccessControl` - `Optional userName` The username to use for authentication. - `class CloudJiraDataSource:` Cloud Jira Data Source integrating JiraReader. - `String authenticationMechanism` Type of Authentication for connecting to Jira APIs. - `String query` JQL (Jira Query Language) query to search. - `Optional apiToken` The API/ Access Token used for Basic, PAT and OAuth2 authentication. - `Optional className` - `Optional cloudId` The cloud ID, used in case of OAuth2. - `Optional email` The email address to use for authentication. - `Optional serverUrl` The server url for Jira Cloud. - `Optional supportsAccessControl` - `class CloudJiraDataSourceV2:` Cloud Jira Data Source integrating JiraReaderV2. - `String authenticationMechanism` Type of Authentication for connecting to Jira APIs. - `String query` JQL (Jira Query Language) query to search. - `String serverUrl` The server url for Jira Cloud. - `Optional apiToken` The API Access Token used for Basic, PAT and OAuth2 authentication. - `Optional apiVersion` Jira REST API version to use (2 or 3). 3 supports Atlassian Document Format (ADF). - `_2("2")` - `_3("3")` - `Optional className` - `Optional cloudId` The cloud ID, used in case of OAuth2. - `Optional email` The email address to use for authentication. - `Optional expand` Fields to expand in the response. - `Optional> fields` List of fields to retrieve from Jira. If None, retrieves all fields. - `Optional getPermissions` Whether to fetch project role permissions and issue-level security - `Optional requestsPerMinute` Rate limit for Jira API requests per minute. - `Optional supportsAccessControl` - `class CloudBoxDataSource:` - `AuthenticationMechanism authenticationMechanism` The type of authentication to use (Developer Token or CCG) - `DEVELOPER_TOKEN("developer_token")` - `CCG("ccg")` - `Optional className` - `Optional clientId` Box API key used for identifying the application the user is authenticating with - `Optional clientSecret` Box API secret used for making auth requests. - `Optional developerToken` Developer token for authentication if authentication_mechanism is 'developer_token'. - `Optional enterpriseId` Box Enterprise ID, if provided authenticates as service. - `Optional folderId` The ID of the Box folder to read from. - `Optional supportsAccessControl` - `Optional userId` Box User ID, if provided authenticates as user. - `String name` The name of the data source. - `String projectId` - `SourceType sourceType` - `S3("S3")` - `AZURE_STORAGE_BLOB("AZURE_STORAGE_BLOB")` - `GOOGLE_DRIVE("GOOGLE_DRIVE")` - `MICROSOFT_ONEDRIVE("MICROSOFT_ONEDRIVE")` - `MICROSOFT_SHAREPOINT("MICROSOFT_SHAREPOINT")` - `SLACK("SLACK")` - `NOTION_PAGE("NOTION_PAGE")` - `CONFLUENCE("CONFLUENCE")` - `JIRA("JIRA")` - `JIRA_V2("JIRA_V2")` - `BOX("BOX")` - `Optional createdAt` Creation datetime - `Optional customMetadata` Custom metadata that will be present on all data loaded from the data source - `class UnionMember0:` - `List` - `String` - `double` - `boolean` - `Optional updatedAt` Update datetime - `Optional versionMetadata` Version metadata for the data source - `Optional readerVersion` The version of the reader to use for this data source. - `_1_0("1.0")` - `_2_0("2.0")` - `_2_1("2.1")` ### Example ```java package com.llamacloud_prod.api.example; import com.llamacloud_prod.api.client.LlamaCloudClient; import com.llamacloud_prod.api.client.okhttp.LlamaCloudOkHttpClient; import com.llamacloud_prod.api.models.datasources.DataSource; import com.llamacloud_prod.api.models.datasources.DataSourceGetParams; public final class Main { private Main() {} public static void main(String[] args) { LlamaCloudClient client = LlamaCloudOkHttpClient.fromEnv(); DataSource dataSource = client.dataSources().get("182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e"); } } ``` #### Response ```json { "id": "182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e", "component": { "foo": "bar" }, "name": "name", "project_id": "182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e", "source_type": "S3", "created_at": "2019-12-27T18:11:19.117Z", "custom_metadata": { "foo": { "foo": "bar" } }, "updated_at": "2019-12-27T18:11:19.117Z", "version_metadata": { "reader_version": "1.0" } } ``` ## Update Data Source `DataSource dataSources().update(DataSourceUpdateParamsparams, RequestOptionsrequestOptions = RequestOptions.none())` **put** `/api/v1/data-sources/{data_source_id}` Update a data source by ID. ### Parameters - `DataSourceUpdateParams params` - `Optional dataSourceId` - `SourceType sourceType` - `S3("S3")` - `AZURE_STORAGE_BLOB("AZURE_STORAGE_BLOB")` - `GOOGLE_DRIVE("GOOGLE_DRIVE")` - `MICROSOFT_ONEDRIVE("MICROSOFT_ONEDRIVE")` - `MICROSOFT_SHAREPOINT("MICROSOFT_SHAREPOINT")` - `SLACK("SLACK")` - `NOTION_PAGE("NOTION_PAGE")` - `CONFLUENCE("CONFLUENCE")` - `JIRA("JIRA")` - `JIRA_V2("JIRA_V2")` - `BOX("BOX")` - `Optional component` Component that implements the data source - `class UnionMember0:` - `class CloudS3DataSource:` - `String bucket` The name of the S3 bucket to read from. - `Optional awsAccessId` The AWS access ID to use for authentication. - `Optional awsAccessSecret` The AWS access secret to use for authentication. - `Optional className` - `Optional prefix` The prefix of the S3 objects to read from. - `Optional regexPattern` The regex pattern to filter S3 objects. Must be a valid regex pattern. - `Optional s3EndpointUrl` The S3 endpoint URL to use for authentication. - `Optional supportsAccessControl` - `class CloudAzStorageBlobDataSource:` - `String accountUrl` The Azure Storage Blob account URL to use for authentication. - `String containerName` The name of the Azure Storage Blob container to read from. - `Optional accountKey` The Azure Storage Blob account key to use for authentication. - `Optional accountName` The Azure Storage Blob account name to use for authentication. - `Optional blob` The blob name to read from. - `Optional className` - `Optional clientId` The Azure AD client ID to use for authentication. - `Optional clientSecret` The Azure AD client secret to use for authentication. - `Optional prefix` The prefix of the Azure Storage Blob objects to read from. - `Optional supportsAccessControl` - `Optional tenantId` The Azure AD tenant ID to use for authentication. - `class CloudGoogleDriveDataSource:` - `String folderId` The ID of the Google Drive folder to read from. - `Optional className` - `Optional serviceAccountKey` A dictionary containing secret values - `Optional supportsAccessControl` - `class CloudOneDriveDataSource:` - `String clientId` The client ID to use for authentication. - `String clientSecret` The client secret to use for authentication. - `String tenantId` The tenant ID to use for authentication. - `String userPrincipalName` The user principal name to use for authentication. - `Optional className` - `Optional folderId` The ID of the OneDrive folder to read from. - `Optional folderPath` The path of the OneDrive folder to read from. - `Optional> requiredExts` The list of required file extensions. - `Optional supportsAccessControl` - `TRUE(true)` - `class CloudSharepointDataSource:` - `String clientId` The client ID to use for authentication. - `String clientSecret` The client secret to use for authentication. - `String tenantId` The tenant ID to use for authentication. - `Optional className` - `Optional driveName` The name of the Sharepoint drive to read from. - `Optional> excludePathPatterns` List of regex patterns for file paths to exclude. Files whose paths (including filename) match any pattern will be excluded. Example: ['/temp/', '/backup/', '.git/', '.tmp$', '^~'] - `Optional folderId` The ID of the Sharepoint folder to read from. - `Optional folderPath` The path of the Sharepoint folder to read from. - `Optional getPermissions` Whether to get permissions for the sharepoint site. - `Optional> includePathPatterns` List of regex patterns for file paths to include. Full paths (including filename) must match at least one pattern to be included. Example: ['/reports/', '/docs/.*.pdf$', '^Report.*.pdf$'] - `Optional> requiredExts` The list of required file extensions. - `Optional siteId` The ID of the SharePoint site to download from. - `Optional siteName` The name of the SharePoint site to download from. - `Optional supportsAccessControl` - `TRUE(true)` - `class CloudSlackDataSource:` - `String slackToken` Slack Bot Token. - `Optional channelIds` Slack Channel. - `Optional channelPatterns` Slack Channel name pattern. - `Optional className` - `Optional earliestDate` Earliest date. - `Optional earliestDateTimestamp` Earliest date timestamp. - `Optional latestDate` Latest date. - `Optional latestDateTimestamp` Latest date timestamp. - `Optional supportsAccessControl` - `class CloudNotionPageDataSource:` - `String integrationToken` The integration token to use for authentication. - `Optional className` - `Optional databaseIds` The Notion Database Id to read content from. - `Optional pageIds` The Page ID's of the Notion to read from. - `Optional supportsAccessControl` - `class CloudConfluenceDataSource:` - `String authenticationMechanism` Type of Authentication for connecting to Confluence APIs. - `String serverUrl` The server URL of the Confluence instance. - `Optional apiToken` The API token to use for authentication. - `Optional className` - `Optional cql` The CQL query to use for fetching pages. - `Optional failureHandling` Configuration for handling failures during processing. Key-value object controlling failure handling behaviors. Example: { "skip_list_failures": true } Currently supports: - skip_list_failures: Skip failed batches/lists and continue processing - `Optional skipListFailures` Whether to skip failed batches/lists and continue processing - `Optional indexRestrictedPages` Whether to index restricted pages. - `Optional keepMarkdownFormat` Whether to keep the markdown format. - `Optional label` The label to use for fetching pages. - `Optional pageIds` The page IDs of the Confluence to read from. - `Optional spaceKey` The space key to read from. - `Optional supportsAccessControl` - `Optional userName` The username to use for authentication. - `class CloudJiraDataSource:` Cloud Jira Data Source integrating JiraReader. - `String authenticationMechanism` Type of Authentication for connecting to Jira APIs. - `String query` JQL (Jira Query Language) query to search. - `Optional apiToken` The API/ Access Token used for Basic, PAT and OAuth2 authentication. - `Optional className` - `Optional cloudId` The cloud ID, used in case of OAuth2. - `Optional email` The email address to use for authentication. - `Optional serverUrl` The server url for Jira Cloud. - `Optional supportsAccessControl` - `class CloudJiraDataSourceV2:` Cloud Jira Data Source integrating JiraReaderV2. - `String authenticationMechanism` Type of Authentication for connecting to Jira APIs. - `String query` JQL (Jira Query Language) query to search. - `String serverUrl` The server url for Jira Cloud. - `Optional apiToken` The API Access Token used for Basic, PAT and OAuth2 authentication. - `Optional apiVersion` Jira REST API version to use (2 or 3). 3 supports Atlassian Document Format (ADF). - `_2("2")` - `_3("3")` - `Optional className` - `Optional cloudId` The cloud ID, used in case of OAuth2. - `Optional email` The email address to use for authentication. - `Optional expand` Fields to expand in the response. - `Optional> fields` List of fields to retrieve from Jira. If None, retrieves all fields. - `Optional getPermissions` Whether to fetch project role permissions and issue-level security - `Optional requestsPerMinute` Rate limit for Jira API requests per minute. - `Optional supportsAccessControl` - `class CloudBoxDataSource:` - `AuthenticationMechanism authenticationMechanism` The type of authentication to use (Developer Token or CCG) - `DEVELOPER_TOKEN("developer_token")` - `CCG("ccg")` - `Optional className` - `Optional clientId` Box API key used for identifying the application the user is authenticating with - `Optional clientSecret` Box API secret used for making auth requests. - `Optional developerToken` Developer token for authentication if authentication_mechanism is 'developer_token'. - `Optional enterpriseId` Box Enterprise ID, if provided authenticates as service. - `Optional folderId` The ID of the Box folder to read from. - `Optional supportsAccessControl` - `Optional userId` Box User ID, if provided authenticates as user. - `Optional customMetadata` Custom metadata that will be present on all data loaded from the data source - `class UnionMember0:` - `List` - `String` - `double` - `boolean` - `Optional name` The name of the data source. ### Returns - `class DataSource:` Schema for a data source. - `String id` Unique identifier - `Component component` Component that implements the data source - `class UnionMember0:` - `class CloudS3DataSource:` - `String bucket` The name of the S3 bucket to read from. - `Optional awsAccessId` The AWS access ID to use for authentication. - `Optional awsAccessSecret` The AWS access secret to use for authentication. - `Optional className` - `Optional prefix` The prefix of the S3 objects to read from. - `Optional regexPattern` The regex pattern to filter S3 objects. Must be a valid regex pattern. - `Optional s3EndpointUrl` The S3 endpoint URL to use for authentication. - `Optional supportsAccessControl` - `class CloudAzStorageBlobDataSource:` - `String accountUrl` The Azure Storage Blob account URL to use for authentication. - `String containerName` The name of the Azure Storage Blob container to read from. - `Optional accountKey` The Azure Storage Blob account key to use for authentication. - `Optional accountName` The Azure Storage Blob account name to use for authentication. - `Optional blob` The blob name to read from. - `Optional className` - `Optional clientId` The Azure AD client ID to use for authentication. - `Optional clientSecret` The Azure AD client secret to use for authentication. - `Optional prefix` The prefix of the Azure Storage Blob objects to read from. - `Optional supportsAccessControl` - `Optional tenantId` The Azure AD tenant ID to use for authentication. - `class CloudGoogleDriveDataSource:` - `String folderId` The ID of the Google Drive folder to read from. - `Optional className` - `Optional serviceAccountKey` A dictionary containing secret values - `Optional supportsAccessControl` - `class CloudOneDriveDataSource:` - `String clientId` The client ID to use for authentication. - `String clientSecret` The client secret to use for authentication. - `String tenantId` The tenant ID to use for authentication. - `String userPrincipalName` The user principal name to use for authentication. - `Optional className` - `Optional folderId` The ID of the OneDrive folder to read from. - `Optional folderPath` The path of the OneDrive folder to read from. - `Optional> requiredExts` The list of required file extensions. - `Optional supportsAccessControl` - `TRUE(true)` - `class CloudSharepointDataSource:` - `String clientId` The client ID to use for authentication. - `String clientSecret` The client secret to use for authentication. - `String tenantId` The tenant ID to use for authentication. - `Optional className` - `Optional driveName` The name of the Sharepoint drive to read from. - `Optional> excludePathPatterns` List of regex patterns for file paths to exclude. Files whose paths (including filename) match any pattern will be excluded. Example: ['/temp/', '/backup/', '.git/', '.tmp$', '^~'] - `Optional folderId` The ID of the Sharepoint folder to read from. - `Optional folderPath` The path of the Sharepoint folder to read from. - `Optional getPermissions` Whether to get permissions for the sharepoint site. - `Optional> includePathPatterns` List of regex patterns for file paths to include. Full paths (including filename) must match at least one pattern to be included. Example: ['/reports/', '/docs/.*.pdf$', '^Report.*.pdf$'] - `Optional> requiredExts` The list of required file extensions. - `Optional siteId` The ID of the SharePoint site to download from. - `Optional siteName` The name of the SharePoint site to download from. - `Optional supportsAccessControl` - `TRUE(true)` - `class CloudSlackDataSource:` - `String slackToken` Slack Bot Token. - `Optional channelIds` Slack Channel. - `Optional channelPatterns` Slack Channel name pattern. - `Optional className` - `Optional earliestDate` Earliest date. - `Optional earliestDateTimestamp` Earliest date timestamp. - `Optional latestDate` Latest date. - `Optional latestDateTimestamp` Latest date timestamp. - `Optional supportsAccessControl` - `class CloudNotionPageDataSource:` - `String integrationToken` The integration token to use for authentication. - `Optional className` - `Optional databaseIds` The Notion Database Id to read content from. - `Optional pageIds` The Page ID's of the Notion to read from. - `Optional supportsAccessControl` - `class CloudConfluenceDataSource:` - `String authenticationMechanism` Type of Authentication for connecting to Confluence APIs. - `String serverUrl` The server URL of the Confluence instance. - `Optional apiToken` The API token to use for authentication. - `Optional className` - `Optional cql` The CQL query to use for fetching pages. - `Optional failureHandling` Configuration for handling failures during processing. Key-value object controlling failure handling behaviors. Example: { "skip_list_failures": true } Currently supports: - skip_list_failures: Skip failed batches/lists and continue processing - `Optional skipListFailures` Whether to skip failed batches/lists and continue processing - `Optional indexRestrictedPages` Whether to index restricted pages. - `Optional keepMarkdownFormat` Whether to keep the markdown format. - `Optional label` The label to use for fetching pages. - `Optional pageIds` The page IDs of the Confluence to read from. - `Optional spaceKey` The space key to read from. - `Optional supportsAccessControl` - `Optional userName` The username to use for authentication. - `class CloudJiraDataSource:` Cloud Jira Data Source integrating JiraReader. - `String authenticationMechanism` Type of Authentication for connecting to Jira APIs. - `String query` JQL (Jira Query Language) query to search. - `Optional apiToken` The API/ Access Token used for Basic, PAT and OAuth2 authentication. - `Optional className` - `Optional cloudId` The cloud ID, used in case of OAuth2. - `Optional email` The email address to use for authentication. - `Optional serverUrl` The server url for Jira Cloud. - `Optional supportsAccessControl` - `class CloudJiraDataSourceV2:` Cloud Jira Data Source integrating JiraReaderV2. - `String authenticationMechanism` Type of Authentication for connecting to Jira APIs. - `String query` JQL (Jira Query Language) query to search. - `String serverUrl` The server url for Jira Cloud. - `Optional apiToken` The API Access Token used for Basic, PAT and OAuth2 authentication. - `Optional apiVersion` Jira REST API version to use (2 or 3). 3 supports Atlassian Document Format (ADF). - `_2("2")` - `_3("3")` - `Optional className` - `Optional cloudId` The cloud ID, used in case of OAuth2. - `Optional email` The email address to use for authentication. - `Optional expand` Fields to expand in the response. - `Optional> fields` List of fields to retrieve from Jira. If None, retrieves all fields. - `Optional getPermissions` Whether to fetch project role permissions and issue-level security - `Optional requestsPerMinute` Rate limit for Jira API requests per minute. - `Optional supportsAccessControl` - `class CloudBoxDataSource:` - `AuthenticationMechanism authenticationMechanism` The type of authentication to use (Developer Token or CCG) - `DEVELOPER_TOKEN("developer_token")` - `CCG("ccg")` - `Optional className` - `Optional clientId` Box API key used for identifying the application the user is authenticating with - `Optional clientSecret` Box API secret used for making auth requests. - `Optional developerToken` Developer token for authentication if authentication_mechanism is 'developer_token'. - `Optional enterpriseId` Box Enterprise ID, if provided authenticates as service. - `Optional folderId` The ID of the Box folder to read from. - `Optional supportsAccessControl` - `Optional userId` Box User ID, if provided authenticates as user. - `String name` The name of the data source. - `String projectId` - `SourceType sourceType` - `S3("S3")` - `AZURE_STORAGE_BLOB("AZURE_STORAGE_BLOB")` - `GOOGLE_DRIVE("GOOGLE_DRIVE")` - `MICROSOFT_ONEDRIVE("MICROSOFT_ONEDRIVE")` - `MICROSOFT_SHAREPOINT("MICROSOFT_SHAREPOINT")` - `SLACK("SLACK")` - `NOTION_PAGE("NOTION_PAGE")` - `CONFLUENCE("CONFLUENCE")` - `JIRA("JIRA")` - `JIRA_V2("JIRA_V2")` - `BOX("BOX")` - `Optional createdAt` Creation datetime - `Optional customMetadata` Custom metadata that will be present on all data loaded from the data source - `class UnionMember0:` - `List` - `String` - `double` - `boolean` - `Optional updatedAt` Update datetime - `Optional versionMetadata` Version metadata for the data source - `Optional readerVersion` The version of the reader to use for this data source. - `_1_0("1.0")` - `_2_0("2.0")` - `_2_1("2.1")` ### Example ```java package com.llamacloud_prod.api.example; import com.llamacloud_prod.api.client.LlamaCloudClient; import com.llamacloud_prod.api.client.okhttp.LlamaCloudOkHttpClient; import com.llamacloud_prod.api.models.datasources.DataSource; import com.llamacloud_prod.api.models.datasources.DataSourceUpdateParams; public final class Main { private Main() {} public static void main(String[] args) { LlamaCloudClient client = LlamaCloudOkHttpClient.fromEnv(); DataSourceUpdateParams params = DataSourceUpdateParams.builder() .dataSourceId("182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e") .sourceType(DataSourceUpdateParams.SourceType.S3) .build(); DataSource dataSource = client.dataSources().update(params); } } ``` #### Response ```json { "id": "182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e", "component": { "foo": "bar" }, "name": "name", "project_id": "182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e", "source_type": "S3", "created_at": "2019-12-27T18:11:19.117Z", "custom_metadata": { "foo": { "foo": "bar" } }, "updated_at": "2019-12-27T18:11:19.117Z", "version_metadata": { "reader_version": "1.0" } } ``` ## Delete Data Source `dataSources().delete(DataSourceDeleteParamsparams = DataSourceDeleteParams.none(), RequestOptionsrequestOptions = RequestOptions.none())` **delete** `/api/v1/data-sources/{data_source_id}` Delete a data source by ID. ### Parameters - `DataSourceDeleteParams params` - `Optional dataSourceId` ### Example ```java package com.llamacloud_prod.api.example; import com.llamacloud_prod.api.client.LlamaCloudClient; import com.llamacloud_prod.api.client.okhttp.LlamaCloudOkHttpClient; import com.llamacloud_prod.api.models.datasources.DataSourceDeleteParams; public final class Main { private Main() {} public static void main(String[] args) { LlamaCloudClient client = LlamaCloudOkHttpClient.fromEnv(); client.dataSources().delete("182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e"); } } ``` ## Domain Types ### Data Source - `class DataSource:` Schema for a data source. - `String id` Unique identifier - `Component component` Component that implements the data source - `class UnionMember0:` - `class CloudS3DataSource:` - `String bucket` The name of the S3 bucket to read from. - `Optional awsAccessId` The AWS access ID to use for authentication. - `Optional awsAccessSecret` The AWS access secret to use for authentication. - `Optional className` - `Optional prefix` The prefix of the S3 objects to read from. - `Optional regexPattern` The regex pattern to filter S3 objects. Must be a valid regex pattern. - `Optional s3EndpointUrl` The S3 endpoint URL to use for authentication. - `Optional supportsAccessControl` - `class CloudAzStorageBlobDataSource:` - `String accountUrl` The Azure Storage Blob account URL to use for authentication. - `String containerName` The name of the Azure Storage Blob container to read from. - `Optional accountKey` The Azure Storage Blob account key to use for authentication. - `Optional accountName` The Azure Storage Blob account name to use for authentication. - `Optional blob` The blob name to read from. - `Optional className` - `Optional clientId` The Azure AD client ID to use for authentication. - `Optional clientSecret` The Azure AD client secret to use for authentication. - `Optional prefix` The prefix of the Azure Storage Blob objects to read from. - `Optional supportsAccessControl` - `Optional tenantId` The Azure AD tenant ID to use for authentication. - `class CloudGoogleDriveDataSource:` - `String folderId` The ID of the Google Drive folder to read from. - `Optional className` - `Optional serviceAccountKey` A dictionary containing secret values - `Optional supportsAccessControl` - `class CloudOneDriveDataSource:` - `String clientId` The client ID to use for authentication. - `String clientSecret` The client secret to use for authentication. - `String tenantId` The tenant ID to use for authentication. - `String userPrincipalName` The user principal name to use for authentication. - `Optional className` - `Optional folderId` The ID of the OneDrive folder to read from. - `Optional folderPath` The path of the OneDrive folder to read from. - `Optional> requiredExts` The list of required file extensions. - `Optional supportsAccessControl` - `TRUE(true)` - `class CloudSharepointDataSource:` - `String clientId` The client ID to use for authentication. - `String clientSecret` The client secret to use for authentication. - `String tenantId` The tenant ID to use for authentication. - `Optional className` - `Optional driveName` The name of the Sharepoint drive to read from. - `Optional> excludePathPatterns` List of regex patterns for file paths to exclude. Files whose paths (including filename) match any pattern will be excluded. Example: ['/temp/', '/backup/', '.git/', '.tmp$', '^~'] - `Optional folderId` The ID of the Sharepoint folder to read from. - `Optional folderPath` The path of the Sharepoint folder to read from. - `Optional getPermissions` Whether to get permissions for the sharepoint site. - `Optional> includePathPatterns` List of regex patterns for file paths to include. Full paths (including filename) must match at least one pattern to be included. Example: ['/reports/', '/docs/.*.pdf$', '^Report.*.pdf$'] - `Optional> requiredExts` The list of required file extensions. - `Optional siteId` The ID of the SharePoint site to download from. - `Optional siteName` The name of the SharePoint site to download from. - `Optional supportsAccessControl` - `TRUE(true)` - `class CloudSlackDataSource:` - `String slackToken` Slack Bot Token. - `Optional channelIds` Slack Channel. - `Optional channelPatterns` Slack Channel name pattern. - `Optional className` - `Optional earliestDate` Earliest date. - `Optional earliestDateTimestamp` Earliest date timestamp. - `Optional latestDate` Latest date. - `Optional latestDateTimestamp` Latest date timestamp. - `Optional supportsAccessControl` - `class CloudNotionPageDataSource:` - `String integrationToken` The integration token to use for authentication. - `Optional className` - `Optional databaseIds` The Notion Database Id to read content from. - `Optional pageIds` The Page ID's of the Notion to read from. - `Optional supportsAccessControl` - `class CloudConfluenceDataSource:` - `String authenticationMechanism` Type of Authentication for connecting to Confluence APIs. - `String serverUrl` The server URL of the Confluence instance. - `Optional apiToken` The API token to use for authentication. - `Optional className` - `Optional cql` The CQL query to use for fetching pages. - `Optional failureHandling` Configuration for handling failures during processing. Key-value object controlling failure handling behaviors. Example: { "skip_list_failures": true } Currently supports: - skip_list_failures: Skip failed batches/lists and continue processing - `Optional skipListFailures` Whether to skip failed batches/lists and continue processing - `Optional indexRestrictedPages` Whether to index restricted pages. - `Optional keepMarkdownFormat` Whether to keep the markdown format. - `Optional label` The label to use for fetching pages. - `Optional pageIds` The page IDs of the Confluence to read from. - `Optional spaceKey` The space key to read from. - `Optional supportsAccessControl` - `Optional userName` The username to use for authentication. - `class CloudJiraDataSource:` Cloud Jira Data Source integrating JiraReader. - `String authenticationMechanism` Type of Authentication for connecting to Jira APIs. - `String query` JQL (Jira Query Language) query to search. - `Optional apiToken` The API/ Access Token used for Basic, PAT and OAuth2 authentication. - `Optional className` - `Optional cloudId` The cloud ID, used in case of OAuth2. - `Optional email` The email address to use for authentication. - `Optional serverUrl` The server url for Jira Cloud. - `Optional supportsAccessControl` - `class CloudJiraDataSourceV2:` Cloud Jira Data Source integrating JiraReaderV2. - `String authenticationMechanism` Type of Authentication for connecting to Jira APIs. - `String query` JQL (Jira Query Language) query to search. - `String serverUrl` The server url for Jira Cloud. - `Optional apiToken` The API Access Token used for Basic, PAT and OAuth2 authentication. - `Optional apiVersion` Jira REST API version to use (2 or 3). 3 supports Atlassian Document Format (ADF). - `_2("2")` - `_3("3")` - `Optional className` - `Optional cloudId` The cloud ID, used in case of OAuth2. - `Optional email` The email address to use for authentication. - `Optional expand` Fields to expand in the response. - `Optional> fields` List of fields to retrieve from Jira. If None, retrieves all fields. - `Optional getPermissions` Whether to fetch project role permissions and issue-level security - `Optional requestsPerMinute` Rate limit for Jira API requests per minute. - `Optional supportsAccessControl` - `class CloudBoxDataSource:` - `AuthenticationMechanism authenticationMechanism` The type of authentication to use (Developer Token or CCG) - `DEVELOPER_TOKEN("developer_token")` - `CCG("ccg")` - `Optional className` - `Optional clientId` Box API key used for identifying the application the user is authenticating with - `Optional clientSecret` Box API secret used for making auth requests. - `Optional developerToken` Developer token for authentication if authentication_mechanism is 'developer_token'. - `Optional enterpriseId` Box Enterprise ID, if provided authenticates as service. - `Optional folderId` The ID of the Box folder to read from. - `Optional supportsAccessControl` - `Optional userId` Box User ID, if provided authenticates as user. - `String name` The name of the data source. - `String projectId` - `SourceType sourceType` - `S3("S3")` - `AZURE_STORAGE_BLOB("AZURE_STORAGE_BLOB")` - `GOOGLE_DRIVE("GOOGLE_DRIVE")` - `MICROSOFT_ONEDRIVE("MICROSOFT_ONEDRIVE")` - `MICROSOFT_SHAREPOINT("MICROSOFT_SHAREPOINT")` - `SLACK("SLACK")` - `NOTION_PAGE("NOTION_PAGE")` - `CONFLUENCE("CONFLUENCE")` - `JIRA("JIRA")` - `JIRA_V2("JIRA_V2")` - `BOX("BOX")` - `Optional createdAt` Creation datetime - `Optional customMetadata` Custom metadata that will be present on all data loaded from the data source - `class UnionMember0:` - `List` - `String` - `double` - `boolean` - `Optional updatedAt` Update datetime - `Optional versionMetadata` Version metadata for the data source - `Optional readerVersion` The version of the reader to use for this data source. - `_1_0("1.0")` - `_2_0("2.0")` - `_2_1("2.1")` ### Data Source Reader Version Metadata - `class DataSourceReaderVersionMetadata:` - `Optional readerVersion` The version of the reader to use for this data source. - `_1_0("1.0")` - `_2_0("2.0")` - `_2_1("2.1")`