> ## Documentation Index > Fetch the complete documentation index at: https://code.dcycle.io/llms.txt > Use this file to discover all available pages before exploring further. # Bulk Upload Vehicle Consumptions > Get a presigned URL to upload multiple vehicle consumption records via CSV # Bulk Upload Vehicle Consumptions Upload hundreds of vehicle fuel consumption records at once via a CSV file. This endpoint returns a presigned S3 URL so you can upload your file, which will be processed asynchronously. This method is **recommended** for bulk uploads. Consumption records are persisted in the database and emissions are calculated automatically based on fuel type and quantity. ## Upload Flow Make a POST request to `/api/v1/vehicle_consumptions/bulk/csv` with your file name Use the presigned URL to upload your CSV directly to S3 (PUT request) The system processes your file in the background, creating consumption records and calculating emissions Check consumption records with the [List Vehicle Consumptions](/api-docs/vehicle-consumptions/list) endpoint ## Step 1: Request Presigned URL ## Request ### Headers Your API key for authentication **Example:** `sk_live_1234567890abcdef` Your organization UUID **Example:** `ff4adcc7-8172-45fe-9cf1-e90a6de53aa9` Your user UUID **Example:** `a1b2c3d4-e5f6-7890-abcd-ef1234567890` ### Body Parameters Name of the CSV file you're going to upload **Example:** `"2024_vehicle_fuel_records.csv"` ### Response Presigned S3 URL to upload the consumption records file Sanitized file name (alphanumeric only) File UUID for tracking Full S3 key where file will be stored Confirmation message ### Example ```bash cURL theme={"theme":{"light":"github-light","dark":"github-dark"}} curl -X POST "https://api.dcycle.io/api/v1/vehicle_consumptions/bulk/csv" \ -H "Authorization: Bearer ${DCYCLE_API_KEY}" \ -H "x-organization-id: ${DCYCLE_ORG_ID}" \ -H "x-user-id: ${DCYCLE_USER_ID}" \ -H "Content-Type: application/json" \ -d '{ "file_name": "2024_vehicle_fuel_records.csv" }' ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}} import requests import os api_key = os.getenv("DCYCLE_API_KEY") org_id = os.getenv("DCYCLE_ORG_ID") user_id = os.getenv("DCYCLE_USER_ID") headers = { "Authorization": f"Bearer {api_key}", "x-organization-id": org_id, "x-user-id": user_id, "Content-Type": "application/json" } payload = { "file_name": "2024_vehicle_fuel_records.csv" } response = requests.post( "https://api.dcycle.io/api/v1/vehicle_consumptions/bulk/csv", headers=headers, json=payload ) result = response.json() upload_url = result["upload_url"] file_id = result["file_id"] print(f"Upload URL obtained: {upload_url[:50]}...") print(f"File ID: {file_id}") ``` ```javascript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark"}} const axios = require('axios'); const apiKey = process.env.DCYCLE_API_KEY; const orgId = process.env.DCYCLE_ORG_ID; const userId = process.env.DCYCLE_USER_ID; const headers = { 'Authorization': `Bearer ${apiKey}`, 'x-organization-id': orgId, 'x-user-id': userId, 'Content-Type': 'application/json' }; const payload = { file_name: '2024_vehicle_fuel_records.csv' }; axios.post( 'https://api.dcycle.io/api/v1/vehicle_consumptions/bulk/csv', payload, { headers } ) .then(response => { const { upload_url, file_id } = response.data; console.log('Upload URL:', upload_url.substring(0, 50) + '...'); console.log('File ID:', file_id); }) .catch(error => console.error(error)); ``` ### Successful Response ```json theme={"theme":{"light":"github-light","dark":"github-dark"}} { "upload_url": "https://dcycle-vehicle-consumptions.s3.amazonaws.com/files/dcycle/...", "file_name": "2024_vehicle_fuel_records", "file_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "destination_file_key": "files/dcycle/.../2024_vehicle_fuel_records.csv", "message": "Upload pre-signed url generated" } ``` ## Step 2: Upload CSV to S3 Use the presigned URL to upload your CSV file directly to S3: ```bash cURL theme={"theme":{"light":"github-light","dark":"github-dark"}} # The presigned URL already includes authentication parameters curl -X PUT "${UPLOAD_URL}" \ -H "Content-Type: text/csv" \ --data-binary @2024_vehicle_fuel_records.csv ``` ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}} import requests # Read CSV file with open('2024_vehicle_fuel_records.csv', 'rb') as file: csv_data = file.read() # Upload to S3 response = requests.put( upload_url, data=csv_data, headers={'Content-Type': 'text/csv'} ) if response.status_code == 200: print("✅ File uploaded successfully") else: print(f"❌ Error: {response.status_code}") ``` ```javascript JavaScript theme={"theme":{"light":"github-light","dark":"github-dark"}} const fs = require('fs'); const axios = require('axios'); // Read CSV file const csvData = fs.readFileSync('2024_vehicle_fuel_records.csv'); // Upload to S3 axios.put(uploadUrl, csvData, { headers: { 'Content-Type': 'text/csv' } }) .then(() => console.log('✅ File uploaded successfully')) .catch(error => console.error('❌ Error:', error)); ``` The presigned URL expires in **15 minutes**. Make sure to upload the file within that time. ## CSV Format The CSV must follow this structure for creating vehicle consumption records: ```csv theme={"theme":{"light":"github-light","dark":"github-dark"}} vehicle_id,quantity,unit_id,start_date,end_date,custom_id,base_total_spend,currency_unit a1b2c3d4-e5f6-7890-abcd-ef1234567890,45.5,liter-unit-uuid,2024-01-15,2024-01-15,INV-2024-001,68.25,euros_(eur) b2c3d4e5-f6g7-8901-bcde-fg2345678901,52.0,liter-unit-uuid,2024-01-10,2024-01-10,INV-2024-002,78.00,euros_(eur) c3d4e5f6-g7h8-9012-cdef-gh3456789012,38.2,liter-unit-uuid,2024-01-08,2024-01-08,INV-2024-003,, ``` ### Required Columns | Column | Description | Valid Values | Example | | ------------ | ------------------------ | ---------------------------- | ----------------- | | `vehicle_id` | UUID of the vehicle | UUID from `/api/v1/vehicles` | `a1b2c3d4-...` | | `quantity` | Fuel quantity consumed | Positive number | `45.5` | | `unit_id` | UUID of measurement unit | UUID from `/api/v1/units` | `liter-unit-uuid` | | `start_date` | Consumption period start | YYYY-MM-DD format | `2024-01-15` | | `end_date` | Consumption period end | YYYY-MM-DD format | `2024-01-15` | ### Optional Columns | Column | Description | Example | | ------------------ | ------------------------------------------------- | ------------------- | | `custom_id` | External reference ID (fuel card, invoice number) | `FUELCARD-2024-001` | | `base_total_spend` | Total cost of the consumption | `120.50` | | `currency_unit` | Currency name for the spend amount | `euros_(eur)` | ### Column Details **vehicle\_id**: * Must be a valid UUID from your vehicles list * Get valid vehicle IDs: `GET /api/v1/vehicles` * Vehicle must belong to your organization **quantity**: * Must be positive number * No negative values allowed * Precision: up to 6 decimal places **unit\_id**: * Must be a valid UUID from the Units endpoint * Typically liquid units (L, gal) for combustion vehicles * Energy units (kWh) for electric vehicles * Get valid unit IDs: `GET /api/v1/units` **start\_date & end\_date**: * Format: `YYYY-MM-DD` * Must be today or in the past * end\_date must be >= start\_date * Can be same day for single refueling events * Used to track consumption periods **custom\_id** (optional): * External reference for tracking * Examples: fuel card transaction ID, invoice number, receipt ID * Useful for reconciliation with fuel card systems * Max length: 255 characters **base\_total\_spend** (optional): * Total monetary cost of the consumption * Negative values are accepted * Can be provided independently of `currency_unit` **currency\_unit** (optional): * Name of the currency for the spend amount * Must match a valid fiat currency unit name (e.g., `euros_(eur)`, `us_dollar_(usd)`, `british_pound_(gbp)`) * Can be provided independently of `base_total_spend` * Get valid currency names: `GET /api/v1/units?type=fiat_currency` ### Download Template Template with correct format and sample data ## Step 3: Asynchronous Processing Once the CSV is uploaded: 1. **Validation**: The system validates format, vehicle IDs, units, and dates 2. **Processing**: Each row is processed and consumption records are created 3. **Emission Calculation**: CO2e values calculated using vehicle fuel type and emission factors 4. **Status Update**: Records marked as `active` or `error` based on processing result 5. **Notification**: (Coming soon) You'll receive a webhook when complete Processing can take from a few seconds to several minutes depending on file size and number of records. ## Step 4: Verify Results After processing, you can query consumption records for each vehicle: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} # List consumptions for a specific vehicle curl "https://api.dcycle.io/api/v1/vehicle_consumptions/vehicle/{vehicle_id}" \ -H "Authorization: Bearer ${DCYCLE_API_KEY}" \ -H "x-organization-id: ${DCYCLE_ORG_ID}" \ -H "x-user-id: ${DCYCLE_USER_ID}" ``` ## Complete Example Script ```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}} import requests import os from time import sleep class DcycleVehicleConsumptions: def __init__(self, api_key, org_id, user_id): self.api_key = api_key self.org_id = org_id self.user_id = user_id self.base_url = "https://api.dcycle.io" self.headers = { "Authorization": f"Bearer {api_key}", "x-organization-id": org_id, "x-user-id": user_id, "Content-Type": "application/json" } def upload_csv(self, csv_file_path): """Complete CSV upload: presigned URL + upload + verification""" # 1. Get presigned URL file_name = os.path.basename(csv_file_path) response = requests.post( f"{self.base_url}/api/v1/vehicle_consumptions/bulk/csv", headers=self.headers, json={"file_name": file_name} ) response.raise_for_status() upload_url = response.json()["upload_url"] file_id = response.json()["file_id"] print(f"✅ Presigned URL obtained (File ID: {file_id})") # 2. Upload file with open(csv_file_path, 'rb') as f: upload_response = requests.put( upload_url, data=f, headers={'Content-Type': 'text/csv'} ) upload_response.raise_for_status() print(f"✅ File uploaded successfully") # 3. Wait for processing (optional) print("⏳ Processing... (this may take a few minutes)") sleep(30) # Wait 30 seconds print("✅ Processing complete. Verify consumption records by vehicle.") return file_id def get_vehicle_consumptions(self, vehicle_id, page=1, size=50): """Get consumptions for a specific vehicle""" response = requests.get( f"{self.base_url}/api/v1/vehicle_consumptions/vehicle/{vehicle_id}", headers=self.headers, params={"page": page, "size": size} ) response.raise_for_status() return response.json() def check_upload_errors(self, vehicle_ids): """Check for consumption records with errors""" total_errors = 0 for vehicle_id in vehicle_ids: response = requests.get( f"{self.base_url}/api/v1/vehicle_consumptions/vehicle/{vehicle_id}", headers=self.headers, params={"page": 1, "size": 1} ) data = response.json() if data['total_error'] > 0: print(f"⚠️ Vehicle {vehicle_id}: {data['total_error']} errors") total_errors += data['total_error'] if total_errors == 0: print("✅ All consumption records processed successfully") return total_errors # Usage if __name__ == "__main__": client = DcycleVehicleConsumptions( api_key=os.getenv("DCYCLE_API_KEY"), org_id=os.getenv("DCYCLE_ORG_ID"), user_id=os.getenv("DCYCLE_USER_ID") ) # Upload consumptions file_id = client.upload_csv("2024_vehicle_fuel_records.csv") # Check for errors (provide vehicle IDs from your CSV) vehicle_ids = [ "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "b2c3d4e5-f6g7-8901-bcde-fg2345678901" ] errors = client.check_upload_errors(vehicle_ids) ``` ```bash Bash theme={"theme":{"light":"github-light","dark":"github-dark"}} #!/bin/bash # Variables API_KEY="${DCYCLE_API_KEY}" ORG_ID="${DCYCLE_ORG_ID}" USER_ID="${DCYCLE_USER_ID}" CSV_FILE="2024_vehicle_fuel_records.csv" BASE_URL="https://api.dcycle.io" # 1. Get presigned URL echo "📤 Requesting presigned URL..." RESPONSE=$(curl -s -X POST "${BASE_URL}/api/v1/vehicle_consumptions/bulk/csv" \ -H "Authorization: Bearer ${API_KEY}" \ -H "x-organization-id: ${ORG_ID}" \ -H "x-user-id: ${USER_ID}" \ -H "Content-Type: application/json" \ -d "{\"file_name\": \"${CSV_FILE}\"}") UPLOAD_URL=$(echo $RESPONSE | jq -r '.upload_url') FILE_ID=$(echo $RESPONSE | jq -r '.file_id') echo "✅ URL obtained (File ID: ${FILE_ID})" # 2. Upload file echo "📤 Uploading CSV file..." curl -X PUT "${UPLOAD_URL}" \ -H "Content-Type: text/csv" \ --data-binary @"${CSV_FILE}" echo "✅ File uploaded successfully" echo "⏳ File is being processed..." ``` ## Common Errors ### 400 Bad Request - Missing file\_name **Cause:** Field required ```json theme={"theme":{"light":"github-light","dark":"github-dark"}} { "detail": "Field required", "field": "file_name" } ``` **Solution:** Include the `file_name` parameter in the request body. ### 403 Forbidden - URL Expired **Cause:** The presigned URL expired (15 minutes) **Solution:** Request a new presigned URL and upload the file immediately. ### 422 Validation Error - CSV Format **Cause:** The CSV has incorrect format or missing required columns **Solution:** Verify that your CSV has all required columns (`vehicle_id`, `quantity`, `unit_id`, `start_date`, `end_date`) and correct format. ### 422 Validation Error - Invalid Vehicle ID **Cause:** `vehicle_id` doesn't exist or doesn't belong to your organization **Solution:** Use the Vehicles endpoint to get valid vehicle IDs from your fleet. ### 422 Validation Error - Invalid Unit ID **Cause:** `unit_id` doesn't exist or is not compatible with vehicle fuel type **Solution:** Use the Units endpoint to get valid unit IDs. For liquid fuels use `L`, for electric use `kWh`. ### 422 Validation Error - Invalid Date **Cause:** Date is in the future or incorrectly formatted **Solution:** Ensure all dates are today or in the past, and use `YYYY-MM-DD` format. ### 422 Validation Error - End Date Before Start Date **Cause:** `end_date` is earlier than `start_date` **Solution:** Ensure `end_date` >= `start_date`. ## Limits and Recommendations | Limit | Value | | ------------------------ | -------------------------- | | **Maximum file size** | 100 MB | | **Maximum rows per CSV** | 100,000 | | **URL expiration time** | 15 minutes | | **Processing time** | \~1 second per 200 records | For very large datasets (>100k records), consider splitting them into multiple smaller CSVs by vehicle or time period. ## Best Practices ### Data Preparation 1. **Get Vehicle IDs first**: Query your vehicles before creating the CSV 2. **Get Unit IDs**: Ensure unit IDs match vehicle fuel types (L for liquid fuels, kWh for electric) 3. **Validate dates**: All dates must be today or in the past 4. **Use custom\_id**: Include external references (fuel card IDs) for reconciliation 5. **Group by refueling events**: One row per refueling event or period ### Error Handling After upload, check for consumption records with `status: "error"`: ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} def check_for_errors_by_vehicle(vehicle_ids): """Check if any consumption records failed to process""" for vehicle_id in vehicle_ids: response = requests.get( f"https://api.dcycle.io/api/v1/vehicle_consumptions/vehicle/{vehicle_id}", headers=headers, params={"filter_by": "status:eqerror"} ) consumptions = response.json() if consumptions['total'] > 0: print(f"⚠️ Vehicle {vehicle_id}: {consumptions['total']} errors") for consumption in consumptions['items']: print(f" - {consumption.get('start_date', 'N/A')}: {consumption.get('quantity')} {consumption['unit']['name']}") # Check after processing check_for_errors_by_vehicle(["vehicle-uuid-1", "vehicle-uuid-2"]) ``` ### Incremental Updates To add new consumption records: 1. Get latest consumption date for each vehicle 2. Export new records from fuel card system 3. Upload only new records (avoid duplicates) ### Data Sources Common sources for vehicle consumption data: * **Fuel Card Systems**: Transaction exports from providers (Shell, BP, Repsol) * **Telematics**: GPS fleet tracking systems with fuel monitoring * **Manual Logs**: Driver-reported fuel purchases * **Accounting Systems**: Fuel expense records * **Electric Vehicle Charging**: Charging station APIs or apps ## Integration Examples ### Fuel Card System Integration ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} def import_from_fuel_card_system(fuel_card_export_file): """Convert fuel card export to Dcycle format""" import pandas as pd # Read fuel card export df = pd.read_csv(fuel_card_export_file) # Get vehicle mapping (license plate -> vehicle_id) vehicles_response = requests.get( "https://api.dcycle.io/api/v1/vehicles", headers=headers, params={"size": 1000} ) vehicles = vehicles_response.json()['items'] license_to_id = {v['license_plate']: v['id'] for v in vehicles} # Get unit ID for liters units_response = requests.get( "https://api.dcycle.io/api/v1/units", headers=headers, params={"type": "liquid"} ) units = units_response.json() liter_unit = next(u for u in units if u['name'] == 'L') # Transform to Dcycle format dcycle_records = [] for _, row in df.iterrows(): vehicle_id = license_to_id.get(row['license_plate']) if vehicle_id: dcycle_records.append({ 'vehicle_id': vehicle_id, 'quantity': row['liters'], 'unit_id': liter_unit['id'], 'start_date': row['transaction_date'], 'end_date': row['transaction_date'], 'custom_id': row['transaction_id'] }) # Create CSV dcycle_df = pd.DataFrame(dcycle_records) dcycle_df.to_csv('dcycle_consumptions.csv', index=False) print(f"✅ Converted {len(dcycle_records)} fuel card transactions") return 'dcycle_consumptions.csv' ``` ### Electric Vehicle Charging Integration ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} def import_ev_charging_data(charging_data): """Import electric vehicle charging data""" # Get kWh unit units_response = requests.get( "https://api.dcycle.io/api/v1/units", headers=headers, params={"type": "energy"} ) units = units_response.json() kwh_unit = next(u for u in units if u['name'] == 'kWh') # Transform charging data dcycle_records = [] for charge in charging_data: dcycle_records.append({ 'vehicle_id': charge['vehicle_id'], 'quantity': charge['kwh_delivered'], 'unit_id': kwh_unit['id'], 'start_date': charge['charge_date'], 'end_date': charge['charge_date'], 'custom_id': f"CHARGE-{charge['session_id']}" }) return dcycle_records ``` ## Related Endpoints View uploaded consumption records Get vehicle IDs for your fleet Get measurement units for consumptions Learn about API authentication