Skip to main content
Migrate data feeds from the v1 schema to the v2 schemaMigrate data feeds from the v1 schema to the v2 schema
Data feed V1 EOL (end of life) supportData feed V1 is being retired and will no longer be supported. V1 data feed exports will be completely shut down, so plan your migration to V2 now. Contact support@forager.ai for the exact shutdown date.

​
Summary

We are migrating our underlying data feed database to Snowflake, this will enable you to have access to following to features:
  • Access crawled data directly in a Snowflake database shared with your account (updated on a rolling basis)
  • Additional export destinations, we now support sending data feeds to the following destinations:
    • Amazon S3
    • Google Cloud Storage
    • Azure Blob (coming soon)
  • Customizable export job schedules, we can now define any schedule to export your Forager data feed to your destination. Have data sent to your export destination at any frequency (charges do apply).
  • Additional file formats, we now can support the following:
    • JSON (NDJSON format)
    • CSV/TSV
    • Parquet
  • Compression can now be applied to your data feed export files using any of the following:
    • Gzip (default)
    • bzip2
    • Brotli
    • Zstandard
  • Ability to receive net new data only: export only the records updated since the last export.

​
Schema

View the full data feed schema here.

​
File paths and names

View new file path and name formats here

​
Breaking changes:

  1. File export compression, all files are now exported using gzip by default, you do have the option to disable this or select from one of the other supported compression types above.
  2. JSON export format, all JSON files will now use NDJSON (one JSON object per line), instead of an array of JSON objects. See an example person file here, and the row format in Output Data File Details.
  3. File paths are now restructured to allow schema changes and to easily retrieve updated data only. For example: /forager_data_feed/<data_feed_id>/<date_created>/<data_feed_type>_<schema_version>/data_<partition_id>.<file_type>.<compression_type>. See changes here.
  4. Properties no longer supported:
    1. background_picture
    2. is_creator
    3. is_influencer
    4. temporary_status
    5. temporary_emoji_status
    6. accessibility_hashtags
    7. accessibility_text
    8. address
    9. primary_locale
    10. supported_locales
  5. Renamed or retyped properties (v1 data dictionary column → v2 schema field):
    1. VOLUNTEERINGS → volunteering
    2. DOMAIN_TRANCO_RANK: VARCHAR → integer (domain_tranco_rank)
    3. KEYWORDS, TECHNOLOGIES: VARCHAR → array of string (keywords, technologies)
    4. FUNDING_ROUNDS: VARCHAR → array of FundingRound objects (funding_rounds)
Last modified on October 28, 2026