Skip to content
LlamaIndex API Docs
Framework Docs
API Reference

Data Sources

List Data Sources
List<DataSource> dataSources().list(DataSourceListParamsparams = DataSourceListParams.none(), RequestOptionsrequestOptions = RequestOptions.none())
GET/api/v1/data-sources
Create Data Source
DataSource dataSources().create(DataSourceCreateParamsparams, RequestOptionsrequestOptions = RequestOptions.none())
POST/api/v1/data-sources
Get Data Source
DataSource dataSources().get(DataSourceGetParamsparams = DataSourceGetParams.none(), RequestOptionsrequestOptions = RequestOptions.none())
GET/api/v1/data-sources/{data_source_id}
Update Data Source
DataSource dataSources().update(DataSourceUpdateParamsparams, RequestOptionsrequestOptions = RequestOptions.none())
PUT/api/v1/data-sources/{data_source_id}
Delete Data Source
dataSources().delete(DataSourceDeleteParamsparams = DataSourceDeleteParams.none(), RequestOptionsrequestOptions = RequestOptions.none())
DELETE/api/v1/data-sources/{data_source_id}
ModelsExpand Collapse
class DataSource:

Schema for a data source.

String id

Unique identifier

formatuuid
Component component

Component that implements the data source

One of the following:
class UnionMember0:
class CloudS3DataSource:
String bucket

The name of the S3 bucket to read from.

Optional<String> awsAccessId

The AWS access ID to use for authentication.

Optional<String> awsAccessSecret

The AWS access secret to use for authentication.

formatpassword
Optional<String> className
Optional<String> prefix

The prefix of the S3 objects to read from.

Optional<String> regexPattern

The regex pattern to filter S3 objects. Must be a valid regex pattern.

Optional<String> s3EndpointUrl

The S3 endpoint URL to use for authentication.

Optional<Boolean> supportsAccessControl
class CloudAzStorageBlobDataSource:
String accountUrl

The Azure Storage Blob account URL to use for authentication.

String containerName

The name of the Azure Storage Blob container to read from.

Optional<String> accountKey

The Azure Storage Blob account key to use for authentication.

formatpassword
Optional<String> accountName

The Azure Storage Blob account name to use for authentication.

Optional<String> blob

The blob name to read from.

Optional<String> className
Optional<String> clientId

The Azure AD client ID to use for authentication.

Optional<String> clientSecret

The Azure AD client secret to use for authentication.

formatpassword
Optional<String> prefix

The prefix of the Azure Storage Blob objects to read from.

Optional<Boolean> supportsAccessControl
Optional<String> tenantId

The Azure AD tenant ID to use for authentication.

class CloudGoogleDriveDataSource:
String folderId

The ID of the Google Drive folder to read from.

Optional<String> className
Optional<ServiceAccountKey> serviceAccountKey

A dictionary containing secret values

Optional<Boolean> supportsAccessControl
class CloudOneDriveDataSource:
String clientId

The client ID to use for authentication.

String clientSecret

The client secret to use for authentication.

formatpassword
String tenantId

The tenant ID to use for authentication.

String userPrincipalName

The user principal name to use for authentication.

Optional<String> className
Optional<String> folderId

The ID of the OneDrive folder to read from.

Optional<String> folderPath

The path of the OneDrive folder to read from.

Optional<List<String>> requiredExts

The list of required file extensions.

Optional<SupportsAccessControl> supportsAccessControl
class CloudSharepointDataSource:
String clientId

The client ID to use for authentication.

String clientSecret

The client secret to use for authentication.

formatpassword
String tenantId

The tenant ID to use for authentication.

Optional<String> className
Optional<String> driveName

The name of the Sharepoint drive to read from.

Optional<List<String>> excludePathPatterns

List of regex patterns for file paths to exclude. Files whose paths (including filename) match any pattern will be excluded. Example: [‘/temp/’, ‘/backup/’, ‘.git/’, ‘.tmp$’, ’^~’]

Optional<String> folderId

The ID of the Sharepoint folder to read from.

Optional<String> folderPath

The path of the Sharepoint folder to read from.

Optional<Boolean> getPermissions

Whether to get permissions for the sharepoint site.

Optional<List<String>> includePathPatterns

List of regex patterns for file paths to include. Full paths (including filename) must match at least one pattern to be included. Example: [‘/reports/’, ‘/docs/..pdf$’, ‘^Report..pdf$’]

Optional<List<String>> requiredExts

The list of required file extensions.

Optional<String> siteId

The ID of the SharePoint site to download from.

Optional<String> siteName

The name of the SharePoint site to download from.

Optional<SupportsAccessControl> supportsAccessControl
class CloudSlackDataSource:
String slackToken

Slack Bot Token.

formatpassword
Optional<String> channelIds

Slack Channel.

Optional<String> channelPatterns

Slack Channel name pattern.

Optional<String> className
Optional<String> earliestDate

Earliest date.

Optional<Double> earliestDateTimestamp

Earliest date timestamp.

Optional<String> latestDate

Latest date.

Optional<Double> latestDateTimestamp

Latest date timestamp.

Optional<Boolean> supportsAccessControl
class CloudNotionPageDataSource:
String integrationToken

The integration token to use for authentication.

formatpassword
Optional<String> className
Optional<String> databaseIds

The Notion Database Id to read content from.

Optional<String> pageIds

The Page ID’s of the Notion to read from.

Optional<Boolean> supportsAccessControl
class CloudConfluenceDataSource:
String authenticationMechanism

Type of Authentication for connecting to Confluence APIs.

String serverUrl

The server URL of the Confluence instance.

Optional<String> apiToken

The API token to use for authentication.

formatpassword
Optional<String> className
Optional<String> cql

The CQL query to use for fetching pages.

Optional<FailureHandlingConfig> failureHandling

Configuration for handling failures during processing. Key-value object controlling failure handling behaviors.

Example: { “skip_list_failures”: true }

Currently supports:

  • skip_list_failures: Skip failed batches/lists and continue processing
Optional<Boolean> skipListFailures

Whether to skip failed batches/lists and continue processing

Optional<Boolean> indexRestrictedPages

Whether to index restricted pages.

Optional<Boolean> keepMarkdownFormat

Whether to keep the markdown format.

Optional<String> label

The label to use for fetching pages.

Optional<String> pageIds

The page IDs of the Confluence to read from.

Optional<String> spaceKey

The space key to read from.

Optional<Boolean> supportsAccessControl
Optional<String> userName

The username to use for authentication.

class CloudJiraDataSource:

Cloud Jira Data Source integrating JiraReader.

String authenticationMechanism

Type of Authentication for connecting to Jira APIs.

String query

JQL (Jira Query Language) query to search.

Optional<String> apiToken

The API/ Access Token used for Basic, PAT and OAuth2 authentication.

formatpassword
Optional<String> className
Optional<String> cloudId

The cloud ID, used in case of OAuth2.

Optional<String> email

The email address to use for authentication.

Optional<String> serverUrl

The server url for Jira Cloud.

Optional<Boolean> supportsAccessControl
class CloudJiraDataSourceV2:

Cloud Jira Data Source integrating JiraReaderV2.

String authenticationMechanism

Type of Authentication for connecting to Jira APIs.

String query

JQL (Jira Query Language) query to search.

String serverUrl

The server url for Jira Cloud.

Optional<String> apiToken

The API Access Token used for Basic, PAT and OAuth2 authentication.

formatpassword
Optional<ApiVersion> apiVersion

Jira REST API version to use (2 or 3). 3 supports Atlassian Document Format (ADF).

One of the following:
_2("2")
_3("3")
Optional<String> className
Optional<String> cloudId

The cloud ID, used in case of OAuth2.

Optional<String> email

The email address to use for authentication.

Optional<String> expand

Fields to expand in the response.

Optional<List<String>> fields

List of fields to retrieve from Jira. If None, retrieves all fields.

Optional<Boolean> getPermissions

Whether to fetch project role permissions and issue-level security

Optional<Long> requestsPerMinute

Rate limit for Jira API requests per minute.

Optional<Boolean> supportsAccessControl
class CloudBoxDataSource:
AuthenticationMechanism authenticationMechanism

The type of authentication to use (Developer Token or CCG)

One of the following:
DEVELOPER_TOKEN("developer_token")
CCG("ccg")
Optional<String> className
Optional<String> clientId

Box API key used for identifying the application the user is authenticating with

Optional<String> clientSecret

Box API secret used for making auth requests.

formatpassword
Optional<String> developerToken

Developer token for authentication if authentication_mechanism is ‘developer_token’.

formatpassword
Optional<String> enterpriseId

Box Enterprise ID, if provided authenticates as service.

Optional<String> folderId

The ID of the Box folder to read from.

Optional<Boolean> supportsAccessControl
Optional<String> userId

Box User ID, if provided authenticates as user.

String name

The name of the data source.

String projectId
SourceType sourceType
One of the following:
S3("S3")
AZURE_STORAGE_BLOB("AZURE_STORAGE_BLOB")
GOOGLE_DRIVE("GOOGLE_DRIVE")
MICROSOFT_ONEDRIVE("MICROSOFT_ONEDRIVE")
MICROSOFT_SHAREPOINT("MICROSOFT_SHAREPOINT")
SLACK("SLACK")
NOTION_PAGE("NOTION_PAGE")
CONFLUENCE("CONFLUENCE")
JIRA("JIRA")
JIRA_V2("JIRA_V2")
BOX("BOX")
Optional<LocalDateTime> createdAt

Creation datetime

formatdate-time
Optional<CustomMetadata> customMetadata

Custom metadata that will be present on all data loaded from the data source

One of the following:
class UnionMember0:
List<JsonValue>
String
double
boolean
Optional<LocalDateTime> updatedAt

Update datetime

formatdate-time
Optional<DataSourceReaderVersionMetadata> versionMetadata

Version metadata for the data source

Optional<ReaderVersion> readerVersion

The version of the reader to use for this data source.

One of the following:
_1_0("1.0")
_2_0("2.0")
_2_1("2.1")
class DataSourceReaderVersionMetadata:
Optional<ReaderVersion> readerVersion

The version of the reader to use for this data source.

One of the following:
_1_0("1.0")
_2_0("2.0")
_2_1("2.1")