builtin.nosql.nosql.iov2.mongodb.MongoDbInput

MongoDbInput

Get records from MongoDB database.

MongoDbInput can be specified as an inputter in a job configuration via the use parameter:

use: nosql.iov2#MongoDbInput

Examples:

Simple find:

{
    "input": {
        "use": "nosql.iov2#MongoDbInput",
        "database": "analytics",
        "collection": "users",
        "host": "mongodb://USER:PASSWORD@host",
        "find": {
            "name": "USERNAME"
        },
        debug: False
    },
    "output": {}
}

To find by ObjectID:

{
    "input": {
        "use": "nosql.iov2#MongoDbInput",
        "database": "analytics",
        "collection": "users",
        "host": "mongodb://USER:PASSWORD@host",
        "find": {
            "_id": {
                "$oid": "611277960f56ac5d0b9ada68"
            }
        },
        debug: False
    }
}

To find by ISODate:

{
    "input": {
        "use": "nosql.iov2#MongoDbInput",
        "database": "analytics",
        "collection": "users",
        "host": "mongodb://USER:PASSWORD@host",
        "find": {
            "my_date_column": {
                "$date": "1985-04-12T23:20:50.52Z"
            }
        }
    }
}

Aggregate:

{
    "input": {
        "use": "nosql.iov2#MongoDbInput",
        "database": "analytics",
        "collection": "users",
        "host": "mongodb://USER:PASSWORD@host",
        "aggregate": [
            {"$match": {
                 "updated_at": {
                    "$gte": {"$date": "2013-10-01T00:00:00.000Z"}},
            {"$match": { "status": "A" } }
        ]
    },
    "output": {}
}

Upsert:

{
    "input": {
        "use": "nosql.iov2#MongoDbInput",
        "database": "analytics",
        "collection": "users",
        "host": "mongodb://USER:PASSWORD@host",
        "find": {
            "age": {"$gte": 18},
        }
        "upsert": {
            "last_updated_isodate": {
                "$gte": {
                    "$date": "2021-01-25T22:43:01.884000+00:00"}}}
    },
    "output": {}
}

Limit:

{
    "input": {
        "use": "nosql.iov2#MongoDbInput",
        "database": "analytics",
        "collection": "pictures",
        "host": "mongodb://USER:PASSWORD@host",
        "find": {
            "color": "red",
        }
        "limit": 3
    },
    "output": {}
}

type

object

properties

  • host

Host

MongoDB database host URL to connect to

type

string

maxLength

65536

minLength

1

format

uri

  • database

Database

MongoDB database name to connect to

type

string

  • credentials

Credentials

MongoDB database credentials to connect to the db

default

anyOf

type

object

type

string

  • collection

Collection

MongoDB collection name records will be searched in

type

string

  • find

Find

A JSON string to search records.

type

object

default

  • aggregate

Aggregate

A JSON string to aggregate records.

default

anyOf

type

object

type

array

items

  • upsert

Upsert

Upsert object to perform upsert on MongoDB. This can be a:

  • string (e.g. my_date_column)

  • Extended JSON dict:

    1. {“$gte”: {“$date”: “%(timestamp)s”}}

    2. {“$gt”: {“$date”: “2020-04-30T00:00:00.000Z”}}

Please note that if you use the exact value for the $date it should be in ISO-8601 Date/Time Format. The timestamp argument would be auto replaced with the datetime the store was modified last time.

anyOf

type

string

type

object

  • limit

Limit

An integer that determines how many records to fetch from MongoDB

type

integer

exclusiveMinimum

0

default

0

  • debug

Debug

Determines if to log JSON queries sent to the MongoDB

type

boolean

default

True

  • timestamp

Timestamp

A timestamp when the store was last modified.This is populated automatically

type

string

format

date-time