airbyte 0.10.0, Apr 14 25

airbyte 0.10.0 published on Monday, Apr 14, 2025 by airbytehq

airbyte.SourceWebScrapper

Explore with Pulumi AI

airbyte 0.10.0 published on Monday, Apr 14, 2025 by airbytehq

Example Usage

Coming soon!

Coming soon!

Coming soon!

Coming soon!

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.airbyte.SourceWebScrapper;
import com.pulumi.airbyte.SourceWebScrapperArgs;
import com.pulumi.airbyte.inputs.SourceWebScrapperConfigurationArgs;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
    public static void main(String[] args) {
        Pulumi.run(App::stack);
    }

    public static void stack(Context ctx) {
        var mySourceWebscrapper = new SourceWebScrapper("mySourceWebscrapper", SourceWebScrapperArgs.builder()
            .configuration(SourceWebScrapperConfigurationArgs.builder()
                .api_token("...my_api_token...")
                .build())
            .definitionId("0425ef2d-74bd-44b7-b93c-adeb71a2b34d")
            .secretId("...my_secret_id...")
            .workspaceId("926420fb-63ad-45d7-bc5b-7416709b93af")
            .build());

    }
}

resources:
  mySourceWebscrapper:
    type: airbyte:SourceWebScrapper
    properties:
      configuration:
        api_token: '...my_api_token...'
      definitionId: 0425ef2d-74bd-44b7-b93c-adeb71a2b34d
      secretId: '...my_secret_id...'
      workspaceId: 926420fb-63ad-45d7-bc5b-7416709b93af

Create SourceWebScrapper Resource

Resources are created with functions called constructors. To learn more about declaring and configuring resources, see Resources.

Constructor syntax

new SourceWebScrapper(name: string, args: SourceWebScrapperArgs, opts?: CustomResourceOptions);

@overload
def SourceWebScrapper(resource_name: str,
                      args: SourceWebScrapperArgs,
                      opts: Optional[ResourceOptions] = None)

@overload
def SourceWebScrapper(resource_name: str,
                      opts: Optional[ResourceOptions] = None,
                      configuration: Optional[SourceWebScrapperConfigurationArgs] = None,
                      workspace_id: Optional[str] = None,
                      definition_id: Optional[str] = None,
                      name: Optional[str] = None,
                      secret_id: Optional[str] = None)

func NewSourceWebScrapper(ctx *Context, name string, args SourceWebScrapperArgs, opts ...ResourceOption) (*SourceWebScrapper, error)

public SourceWebScrapper(string name, SourceWebScrapperArgs args, CustomResourceOptions? opts = null)

public SourceWebScrapper(String name, SourceWebScrapperArgs args)
public SourceWebScrapper(String name, SourceWebScrapperArgs args, CustomResourceOptions options)

type: airbyte:SourceWebScrapper
properties: # The arguments to resource properties.
options: # Bag of options to control resource's behavior.

Parameters

name string: The unique name of the resource.
args SourceWebScrapperArgs: The arguments to resource properties.
opts CustomResourceOptions: Bag of options to control resource's behavior.

resource_name str: The unique name of the resource.
args SourceWebScrapperArgs: The arguments to resource properties.
opts ResourceOptions: Bag of options to control resource's behavior.

ctx Context: Context object for the current deployment.
name string: The unique name of the resource.
args SourceWebScrapperArgs: The arguments to resource properties.
opts ResourceOption: Bag of options to control resource's behavior.

name string: The unique name of the resource.
args SourceWebScrapperArgs: The arguments to resource properties.
opts CustomResourceOptions: Bag of options to control resource's behavior.

name String: The unique name of the resource.
args SourceWebScrapperArgs: The arguments to resource properties.
options CustomResourceOptions: Bag of options to control resource's behavior.

Constructor example

The following reference example uses placeholder values for all input properties.

var sourceWebScrapperResource = new Airbyte.SourceWebScrapper("sourceWebScrapperResource", new()
{
    Configuration = new Airbyte.Inputs.SourceWebScrapperConfigurationArgs
    {
        ApiToken = "string",
    },
    WorkspaceId = "string",
    DefinitionId = "string",
    Name = "string",
    SecretId = "string",
});

example, err := airbyte.NewSourceWebScrapper(ctx, "sourceWebScrapperResource", &airbyte.SourceWebScrapperArgs{
Configuration: &.SourceWebScrapperConfigurationArgs{
ApiToken: pulumi.String("string"),
},
WorkspaceId: pulumi.String("string"),
DefinitionId: pulumi.String("string"),
Name: pulumi.String("string"),
SecretId: pulumi.String("string"),
})

var sourceWebScrapperResource = new SourceWebScrapper("sourceWebScrapperResource", SourceWebScrapperArgs.builder()
    .configuration(SourceWebScrapperConfigurationArgs.builder()
        .apiToken("string")
        .build())
    .workspaceId("string")
    .definitionId("string")
    .name("string")
    .secretId("string")
    .build());

source_web_scrapper_resource = airbyte.SourceWebScrapper("sourceWebScrapperResource",
    configuration={
        "api_token": "string",
    },
    workspace_id="string",
    definition_id="string",
    name="string",
    secret_id="string")

const sourceWebScrapperResource = new airbyte.SourceWebScrapper("sourceWebScrapperResource", {
    configuration: {
        apiToken: "string",
    },
    workspaceId: "string",
    definitionId: "string",
    name: "string",
    secretId: "string",
});

type: airbyte:SourceWebScrapper
properties:
    configuration:
        apiToken: string
    definitionId: string
    name: string
    secretId: string
    workspaceId: string

SourceWebScrapper Resource Properties

To learn more about resource properties and how to use them, see Inputs and Outputs in the Architecture and Concepts docs.

Inputs

In Python, inputs that are objects can be passed either as argument classes or as dictionary literals.

The SourceWebScrapper resource accepts the following input properties:

Configuration SourceWebScrapperConfiguration
WorkspaceId string
DefinitionId string: The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
Name string: Name of the source e.g. dev-mysql-instance.
SecretId string: Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.

Configuration SourceWebScrapperConfigurationArgs
WorkspaceId string
DefinitionId string: The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
Name string: Name of the source e.g. dev-mysql-instance.
SecretId string: Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.

configuration SourceWebScrapperConfiguration
workspaceId String
definitionId String: The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
name String: Name of the source e.g. dev-mysql-instance.
secretId String: Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.

configuration SourceWebScrapperConfiguration
workspaceId string
definitionId string: The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
name string: Name of the source e.g. dev-mysql-instance.
secretId string: Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.

configuration SourceWebScrapperConfigurationArgs
workspace_id str
definition_id str: The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
name str: Name of the source e.g. dev-mysql-instance.
secret_id str: Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.

configuration Property Map
workspaceId String
definitionId String: The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
name String: Name of the source e.g. dev-mysql-instance.
secretId String: Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.

Outputs

All input properties are implicitly available as output properties. Additionally, the SourceWebScrapper resource produces the following output properties:

CreatedAt double
Id string: The provider-assigned unique ID for this managed resource.
ResourceAllocation SourceWebScrapperResourceAllocation: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
SourceId string
SourceType string

CreatedAt float64
Id string: The provider-assigned unique ID for this managed resource.
ResourceAllocation SourceWebScrapperResourceAllocation: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
SourceId string
SourceType string

createdAt Double
id String: The provider-assigned unique ID for this managed resource.
resourceAllocation SourceWebScrapperResourceAllocation: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
sourceId String
sourceType String

createdAt number
id string: The provider-assigned unique ID for this managed resource.
resourceAllocation SourceWebScrapperResourceAllocation: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
sourceId string
sourceType string

created_at float
id str: The provider-assigned unique ID for this managed resource.
resource_allocation SourceWebScrapperResourceAllocation: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
source_id str
source_type str

createdAt Number
id String: The provider-assigned unique ID for this managed resource.
resourceAllocation Property Map: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
sourceId String
sourceType String

Look up Existing SourceWebScrapper Resource

Get an existing SourceWebScrapper resource’s state with the given name, ID, and optional extra properties used to qualify the lookup.

public static get(name: string, id: Input<ID>, state?: SourceWebScrapperState, opts?: CustomResourceOptions): SourceWebScrapper

@staticmethod
def get(resource_name: str,
        id: str,
        opts: Optional[ResourceOptions] = None,
        configuration: Optional[SourceWebScrapperConfigurationArgs] = None,
        created_at: Optional[float] = None,
        definition_id: Optional[str] = None,
        name: Optional[str] = None,
        resource_allocation: Optional[SourceWebScrapperResourceAllocationArgs] = None,
        secret_id: Optional[str] = None,
        source_id: Optional[str] = None,
        source_type: Optional[str] = None,
        workspace_id: Optional[str] = None) -> SourceWebScrapper

func GetSourceWebScrapper(ctx *Context, name string, id IDInput, state *SourceWebScrapperState, opts ...ResourceOption) (*SourceWebScrapper, error)

public static SourceWebScrapper Get(string name, Input<string> id, SourceWebScrapperState? state, CustomResourceOptions? opts = null)

public static SourceWebScrapper get(String name, Output<String> id, SourceWebScrapperState state, CustomResourceOptions options)

resources:  _:    type: airbyte:SourceWebScrapper    get:      id: ${id}

name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.
state: Any extra arguments used during the lookup.
opts: A bag of options that control this resource's behavior.

resource_name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.

name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.
state: Any extra arguments used during the lookup.
opts: A bag of options that control this resource's behavior.

name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.
state: Any extra arguments used during the lookup.
opts: A bag of options that control this resource's behavior.

name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.
state: Any extra arguments used during the lookup.
opts: A bag of options that control this resource's behavior.

The following state arguments are supported:

Configuration SourceWebScrapperConfiguration
CreatedAt double
DefinitionId string: The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
Name string: Name of the source e.g. dev-mysql-instance.
ResourceAllocation SourceWebScrapperResourceAllocation: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
SecretId string: Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.
SourceId string
SourceType string
WorkspaceId string

Configuration SourceWebScrapperConfigurationArgs
CreatedAt float64
DefinitionId string: The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
Name string: Name of the source e.g. dev-mysql-instance.
ResourceAllocation SourceWebScrapperResourceAllocationArgs: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
SecretId string: Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.
SourceId string
SourceType string
WorkspaceId string

configuration SourceWebScrapperConfiguration
createdAt Double
definitionId String: The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
name String: Name of the source e.g. dev-mysql-instance.
resourceAllocation SourceWebScrapperResourceAllocation: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
secretId String: Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.
sourceId String
sourceType String
workspaceId String

configuration SourceWebScrapperConfiguration
createdAt number
definitionId string: The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
name string: Name of the source e.g. dev-mysql-instance.
resourceAllocation SourceWebScrapperResourceAllocation: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
secretId string: Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.
sourceId string
sourceType string
workspaceId string

configuration SourceWebScrapperConfigurationArgs
created_at float
definition_id str: The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
name str: Name of the source e.g. dev-mysql-instance.
resource_allocation SourceWebScrapperResourceAllocationArgs: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
secret_id str: Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.
source_id str
source_type str
workspace_id str

configuration Property Map
createdAt Number
definitionId String: The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
name String: Name of the source e.g. dev-mysql-instance.
resourceAllocation Property Map: actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level.
secretId String: Optional secretID obtained through the public API OAuth redirect flow. Requires replacement if changed.
sourceId String
sourceType String
workspaceId String

Supporting Types

SourceWebScrapperConfiguration, SourceWebScrapperConfigurationArgs

ApiToken string: API token to use. Find it at https://cloud.webscraper.io/api

ApiToken string: API token to use. Find it at https://cloud.webscraper.io/api

apiToken String: API token to use. Find it at https://cloud.webscraper.io/api

apiToken string: API token to use. Find it at https://cloud.webscraper.io/api

api_token str: API token to use. Find it at https://cloud.webscraper.io/api

apiToken String: API token to use. Find it at https://cloud.webscraper.io/api

SourceWebScrapperResourceAllocation, SourceWebScrapperResourceAllocationArgs

Default SourceWebScrapperResourceAllocationDefault: optional resource requirements to run workers (blank for unbounded allocations)
JobSpecifics List<SourceWebScrapperResourceAllocationJobSpecific>

Default SourceWebScrapperResourceAllocationDefault: optional resource requirements to run workers (blank for unbounded allocations)
JobSpecifics []SourceWebScrapperResourceAllocationJobSpecific

default_ SourceWebScrapperResourceAllocationDefault: optional resource requirements to run workers (blank for unbounded allocations)
jobSpecifics List<SourceWebScrapperResourceAllocationJobSpecific>

default SourceWebScrapperResourceAllocationDefault: optional resource requirements to run workers (blank for unbounded allocations)
jobSpecifics SourceWebScrapperResourceAllocationJobSpecific[]

default SourceWebScrapperResourceAllocationDefault: optional resource requirements to run workers (blank for unbounded allocations)
job_specifics Sequence[SourceWebScrapperResourceAllocationJobSpecific]

default Property Map: optional resource requirements to run workers (blank for unbounded allocations)
jobSpecifics List<Property Map>

SourceWebScrapperResourceAllocationDefault, SourceWebScrapperResourceAllocationDefaultArgs

CpuLimit string
CpuRequest string
EphemeralStorageLimit string
EphemeralStorageRequest string
MemoryLimit string
MemoryRequest string

CpuLimit string
CpuRequest string
EphemeralStorageLimit string
EphemeralStorageRequest string
MemoryLimit string
MemoryRequest string

cpuLimit String
cpuRequest String
ephemeralStorageLimit String
ephemeralStorageRequest String
memoryLimit String
memoryRequest String

cpuLimit string
cpuRequest string
ephemeralStorageLimit string
ephemeralStorageRequest string
memoryLimit string
memoryRequest string

cpu_limit str
cpu_request str
ephemeral_storage_limit str
ephemeral_storage_request str
memory_limit str
memory_request str

cpuLimit String
cpuRequest String
ephemeralStorageLimit String
ephemeralStorageRequest String
memoryLimit String
memoryRequest String

SourceWebScrapperResourceAllocationJobSpecific, SourceWebScrapperResourceAllocationJobSpecificArgs

JobType string: enum that describes the different types of jobs that the platform runs. must be one of ["getspec", "checkconnection", "discoverschema", "sync", "resetconnection", "connection_updater", "replicate"]
ResourceRequirements SourceWebScrapperResourceAllocationJobSpecificResourceRequirements: optional resource requirements to run workers (blank for unbounded allocations)

JobType string: enum that describes the different types of jobs that the platform runs. must be one of ["getspec", "checkconnection", "discoverschema", "sync", "resetconnection", "connection_updater", "replicate"]
ResourceRequirements SourceWebScrapperResourceAllocationJobSpecificResourceRequirements: optional resource requirements to run workers (blank for unbounded allocations)

jobType String: enum that describes the different types of jobs that the platform runs. must be one of ["getspec", "checkconnection", "discoverschema", "sync", "resetconnection", "connection_updater", "replicate"]
resourceRequirements SourceWebScrapperResourceAllocationJobSpecificResourceRequirements: optional resource requirements to run workers (blank for unbounded allocations)

jobType string: enum that describes the different types of jobs that the platform runs. must be one of ["getspec", "checkconnection", "discoverschema", "sync", "resetconnection", "connection_updater", "replicate"]
resourceRequirements SourceWebScrapperResourceAllocationJobSpecificResourceRequirements: optional resource requirements to run workers (blank for unbounded allocations)

job_type str: enum that describes the different types of jobs that the platform runs. must be one of ["getspec", "checkconnection", "discoverschema", "sync", "resetconnection", "connection_updater", "replicate"]
resource_requirements SourceWebScrapperResourceAllocationJobSpecificResourceRequirements: optional resource requirements to run workers (blank for unbounded allocations)

jobType String: enum that describes the different types of jobs that the platform runs. must be one of ["getspec", "checkconnection", "discoverschema", "sync", "resetconnection", "connection_updater", "replicate"]
resourceRequirements Property Map: optional resource requirements to run workers (blank for unbounded allocations)

SourceWebScrapperResourceAllocationJobSpecificResourceRequirements, SourceWebScrapperResourceAllocationJobSpecificResourceRequirementsArgs

CpuLimit string
CpuRequest string
EphemeralStorageLimit string
EphemeralStorageRequest string
MemoryLimit string
MemoryRequest string

CpuLimit string
CpuRequest string
EphemeralStorageLimit string
EphemeralStorageRequest string
MemoryLimit string
MemoryRequest string

cpuLimit String
cpuRequest String
ephemeralStorageLimit String
ephemeralStorageRequest String
memoryLimit String
memoryRequest String

cpuLimit string
cpuRequest string
ephemeralStorageLimit string
ephemeralStorageRequest string
memoryLimit string
memoryRequest string

cpu_limit str
cpu_request str
ephemeral_storage_limit str
ephemeral_storage_request str
memory_limit str
memory_request str

cpuLimit String
cpuRequest String
ephemeralStorageLimit String
ephemeralStorageRequest String
memoryLimit String
memoryRequest String

Import

$ pulumi import airbyte:index/sourceWebScrapper:SourceWebScrapper my_airbyte_source_web_scrapper ""

To learn more about importing existing cloud resources, see Importing resources.

Package Details

Repository: airbyte airbytehq/terraform-provider-airbyte
License
Notes: This Pulumi package is based on the airbyte Terraform Provider.

airbyte 0.10.0 published on Monday, Apr 14, 2025 by airbytehq

airbytehq/terraform-provider-airbyte

airbyte.SourceWebScrapper

On this page

On this page

Example Usage

Create SourceWebScrapper Resource

Constructor syntax

Parameters

Constructor example

SourceWebScrapper Resource Properties

Inputs

Outputs

Look up Existing SourceWebScrapper Resource

Supporting Types

SourceWebScrapperConfiguration, SourceWebScrapperConfigurationArgs

SourceWebScrapperResourceAllocation, SourceWebScrapperResourceAllocationArgs

SourceWebScrapperResourceAllocationDefault, SourceWebScrapperResourceAllocationDefaultArgs

SourceWebScrapperResourceAllocationJobSpecific, SourceWebScrapperResourceAllocationJobSpecificArgs

SourceWebScrapperResourceAllocationJobSpecificResourceRequirements, SourceWebScrapperResourceAllocationJobSpecificResourceRequirementsArgs

Import

Package Details

On this page

On this page