providers section

providers:
  provider_name:
    !provider_type
      [parameters]

Providers are the means of providing data to an endpoint, including using data from the response of one endpoint in the request of another. The way providers handle data can be thought of as a FIFO queue--when an endpoint uses data from a provider it "pops" a value from the beginning of the queue and when an endpoint provides data to a provider it is "pushed" to the end of the queue. Every provider has an internal buffer with has a soft limit on how many items can be stored.

A provider_name is any valid js identifier, except for "request", "response", "stats" and "for_each", (as well as "null"), which are reserved.

Example:

providers:
  session: !response
    auto_return: force
  username: !file
    path: "usernames.csv"
    repeat: true

There are four provider_types: file, response, list and range.

file

The file provider_type reads data from a file. Every line in the file is read as a value. A file provider has the following parameters:

  • path - A V-Template value indicating the path to the file on the file system. When a relative path is specified it is interpreted as relative to the config file. Absolute paths are supported though discouraged as they prevent the config file from being platform agnostic.

  • repeat - Optional A boolean value which when true indicates when the provider file provider gets to the end of the file it should start back at the beginning. Defaults to false.

  • unique - Optional A boolean value which when true makes the provider a "unique" provider--meaning each item within the provider will be a unique JSON value without duplicates. Defaults to false.

  • auto_return Optional - This parameter specifies that when this provider is used by a request, after a response is received the value is automatically returned to the provider. Valid options for this parameter are block, force, and if_not_full. See the send parameter under the endpoints.provides subsection for details on the effect of these options.

  • buffer Optional - Specifies the soft limit for a provider's buffer. This can be indicated with an integer greater than zero or the value auto. The value auto indicates that the soft limit can increase as needed. This happens after a provider is full then later becomes empty. Defaults to auto.

  • format Optional - Specifies the format for the file. The format can be one of line (the default), json, or csv.

    The line format will read the file one line at a time with each line ending in a newline (\n) or a carriage return and a newline (\r\n). Every line will attempt to be parsed as JSON, but if it is not valid JSON it will be a string. Note that a JSON object which spans multiple lines in the file, for example, will not parse into a single object.

    The json format will read the file as a stream of JSON values. Every JSON value must be self-delineating (an object, array or string), or must be separated by whitespace or a self-delineating value. For example, the following:

    {"a":1}{"foo":"bar"}47[1,2,3]"some text"true 56

    Would parse into separate JSON values of {"a": 1}, {"foo": "bar"}, 47, [1, 2, 3], "some text", true, and 56.

    The csv format will read the file as a CSV file. Every non-header column will attempt to be parsed as JSON, but if it is not valid JSON it will be a string.

    For the !csv variant, the following sub-parameters are available:

    Sub-parameter Description

    comment Optional

    Specifies a single-byte character which will mark a CSV record as a comment (ex. #). When not specified, no character is treated as a comment.

    delimiter Optional

    Specifies a single-byte character used to separate columns in a record. Defaults to comma (,).

    double_quote Optional

    A boolean that when enabled makes it so two quote characters can be used to escape quotes within a column. Defaults to true.

    escape Optional

    Specifies a single-byte character which will be used to escape nested quote characters (ex. \). When not specified, escapes are disabled.

    headers Optional

    Can be either a boolean value or a string array. When a boolean, it indicates whether the first row in the file should be interpreted as column headers. When an array, the values are used as the column header names.

    When headers are specified, each record served from the file will use the headers as keys for each column. When no headers are specified (the default), then each record will be returned as an array of values.

    For example, with the following CSV file:

    id,name
0,Fred
1,Wilma
2,Pebbles

    If headers was true then the following values would be provided (shown in JSON syntax): {"id": 0, name: "Fred"}, {"id": 1, name: "Wilma"}, and {"id": 3, name: "Pebbles"}.

    If headers was false then the following values would be provided: [0, "Fred"], [1, "Wilma"], and [2, "Pebbles"].

    If headers was - foo\n- bar then the following values would be provided: {"foo": "id", "bar": "name"}, {"foo": 0, "bar": "Fred"}, {"foo": 1, "bar": "Wilma"}, and {"foo": 3, "bar": "Pebbles"}.

    terminator Optional

    Specifies a single-byte character used to terminate each record in the CSV file. Defaults to a special value where \r, \n, and \r\n are all accepted as terminators.

    When specified, Pewpew becomes self-aware, unfolding a series of events which will ultimately lead to the end of the human race.

    quote Optional

    Specifies a single-byte character that will be used to quote CSV columns. Defaults to the double-quote character (").

  • random Optional - A boolean indicating that each record in the file should be returned in random order. Defaults to false.

    When enabled there is no sense of "fairness" in the randomization. Any record in the file could be used more than once before other records are used.

Example, the following:

providers:
  username: !file
    path: "usernames.csv"
    format: !line
    repeat: true
    random: true

response

Unlike other provider_types response does not automatically receive data from a source. Instead a response provider is available to be a "sink" for data originating from an HTTP response. The response provider has the following parameters.

  • auto_return Optional - This parameter specifies that when this provider is used and an individual endpoint call concludes, the value it got from this provider should be sent back to the provider. Valid options for this parameter are block, force, and if_not_full. See the send parameter under the endpoints.provides subsection for details on the effect of these options.
  • buffer Optional - Specifies the soft limit for a provider's buffer. This can be indicated with an integer greater than zero or the value auto. The value auto indicates that if the provider's buffer becomes empty it will automatically increase the buffer size to help prevent the provider from becoming empty again in the future. Defaults to auto.
  • unique - Optional A boolean value which when true makes the provider a "unique" provider--meaning each item within the provider will be a unique JSON value without duplicates. Defaults to false.

Example, the following:

providers:
  session: !response
      buffer: 1000
      auto_return: if_not_full

list

The list provider_type creates a means of specifying an array of static values to be used as a provider. A list provider can be specified in two forms, either implicitly or explicitly. The explicit form has the following parameters:

  • random Optional - A boolean indicating that entries in the values array should provided in random order. When combined with repeat there is no sense of "fairness" in the randomization. Defaults to false.
  • repeat Optional - A boolean indicating that the array should repeat infitely. Defaults to true.
  • values - An array of json values.
  • unique - Optional A boolean value which when true makes the provider a "unique" provider--meaning each item within the provider will be a unique JSON value without duplicates. Defaults to false.

Example, the following:

providers:
  foo: !list
    - 123
    - 456
    - 789

is an example of an implicit list provider. It creates a list provider named foo where the first value provided will be 123, the second 456, third 789 then for subsequent values it will start over at the beginning.

Example, the following:

providers:
  foo: !list
    values:
      - 123
      - 456
      - 789
    random: true

is an example of an explicit list provider. It creates a list provider named foo where the value provided will be randomized between the values listed.

range

The range provider_type provides an incrementing sequence of numbers in a given range. A range provider takes three parameters.

  • start Optional - A whole number in the range of [-9223372036854775808, 9223372036854775807]. This indicates what the starting number should be for the range. Defaults to 0.
  • end Optional - A whole number in the range of [-9223372036854775808, 9223372036854775807]. This indicates what the end number should be for the range. This number is included in the range. Defaults to 9223372036854775807.
  • step Optional - A whole number in the range of [1, 65535]. This indicates how big each "step" in the range will be. Defaults to 1.
  • repeat Optional - A boolean which causes the range to repeat infinitely. Defaults to false.
  • unique - Optional A boolean value which when true makes the provider a "unique" provider--meaning each item within the provider will be a unique JSON value without duplicates. Defaults to false.

Examples:

providers:
  foo: !range

Will use the default settings and foo will provide the values 0, 1, 2, etc. until it yields the end number (9223372036854775807).

providers:
  foo: !range
    start: -50
    end: 100
    step: 2

In this case foo will provide the valuels -50, -48, -46, etc. until it yields 100.

null

A special predefined null provider (named "null") is also available. The null provider endlessly yields null values. Is it equivalent to

providers:
  null: !list
    repeat: true
    values:
      - null

This provider is mainly for sending dummy values to expression functions.