Using 'mongoimport' to Import Data into MongoDB (with examples)

The mongoimport command is a powerful utility provided by MongoDB that allows users to import data from JSON, CSV, or TSV files into a MongoDB database. This makes it incredibly valuable for data migration and integration tasks, where data stored in different formats needs to be imported into a MongoDB collection. The command helps in facilitating continuous data ingestion workflows, ensuring that MongoDB collections are kept up-to-date with relevant data from external sources.

Use Case 1: Import a JSON file into a specific collection

Code:

mongoimport --file=path/to/file.json --uri=mongodb_uri --collection=collection_name

Motivation: One of the primary use cases for mongoimport is to seed initial data from a JSON file into a MongoDB collection. This method is useful during the development phase or when setting up a new environment from scratch.

Explanation:

• --file=path/to/file.json: Designates the path to the JSON file that contains the data to be imported.
• --uri=mongodb_uri: Specifies the complete MongoDB URI, including the database to connect to.
• --collection=collection_name: Identifies the specific collection where the data will be imported.

Example Output: Upon successful import, you might see a console message like: “imported 100 documents”.

Use Case 2: Import a CSV file, using the first line of the file to determine field names

Code:

mongoimport --type=csv --file=path/to/file.csv --db=database_name --collection=collection_name

Motivation: Importing a CSV file is particularly useful when dealing with structured data extracted from spreadsheets or relational databases. The first row of the CSV is expected to contain field names which the MongoDB collection will use to organize the data.

Explanation:

• --type=csv: Informs mongoimport that the file type is CSV.
• --file=path/to/file.csv: Specifies the path to the CSV file.
• --db=database_name: Indicates the database where the collection is located.
• --collection=collection_name: Specifies the collection into which the data will be imported.

Example Output: The terminal will display a message such as: “Connected to: MongoDB at mongodb_uri.” “imported 150 documents”.

Use Case 3: Import a JSON array, using each element as a separate document

Code:

mongoimport --jsonArray --file=path/to/file.json

Motivation: When JSON data is structured as an array, each object within this array should ideally be treated as an individual document in the collection. This command is optimal for situations where bulk data, stored in arrays, needs to be imported efficiently.

Explanation:

• --jsonArray: Specifies that the input file contains a JSON array.
• --file=path/to/file.json: Points to the JSON file that holds the data.

Example Output: Upon successful import, you’ll see output like: “imported 75 documents”.

Use Case 4: Import a JSON file using a specific mode and a query to match existing documents

Code:

mongoimport --file=path/to/file.json --mode=delete|merge|upsert --upsertFields="field1,field2,..."

Motivation: This use case is beneficial for maintaining an up-to-date dataset by either updating existing entries or deleting them based on a specified query. It’s frequently used in situations requiring synchronization between different datasets.

Explanation:

• --file=path/to/file.json: Provides the path to the JSON file.
• --mode=delete|merge|upsert: Determines the operation mode. ‘Delete’ removes matched records, ‘merge’ updates existing, and ‘upsert’ inserts a new record if none match.
• --upsertFields="field1,field2,...": Lists the fields that will be used to match existing documents.

Example Output: Output could include messages like: “upserted 20 documents”, “matched 20”.

Use Case 5: Import a CSV file, reading field names from a separate CSV file and ignoring fields with empty values

Code:

mongoimport --type=csv --file=path/to/file.csv --fieldFile=path/to/field_file.csv --ignoreBlanks

Motivation: This is advantageous when the field names are not included in the CSV file and must be acquired from another source. Simultaneously, it helps clean up data by disregarding undefined fields, thereby maintaining data integrity.

Explanation:

• --type=csv: Specifies that the input type is CSV.
• --file=path/to/file.csv: Denotes the path to the data CSV file.
• --fieldFile=path/to/field_file.csv: Provides the file containing field names.
• --ignoreBlanks: Instructs to skip fields with blank values.

Example Output: You would see a similar output to previous examples, without inserting empty values: “imported 200 documents”.

Conclusion:

The mongoimport command is exceptionally versatile, accommodating various formats and data manipulation requirements. Whether you’re dealing with JSON, CSV, or TSV formats, each use case above demonstrates mongoimport’s ability to simplify the process of importing and updating data into MongoDB collections efficiently. By utilizing its options, developers and data engineers can maintain consistency, integrity, and structure of data within their MongoDB databases.