2025-12-05 20:00:07 [scrapy.utils.log] INFO: Scrapy 2.11.2 started (bot: SourcingV2)
2025-12-05 20:00:07 [scrapy.utils.log] INFO: Versions: lxml 5.2.2.0, libxml2 2.12.6, cssselect 1.2.0, parsel 1.9.1, w3lib 2.1.2, Twisted 24.3.0, Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0], pyOpenSSL 24.1.0 (OpenSSL 3.2.1 30 Jan 2024), cryptography 42.0.7, Platform Linux-5.15.0-1098-azure-x86_64-with-glibc2.36
2025-12-05 20:00:07 [ayvens.es] INFO: Starting spider ayvens.es
2025-12-05 20:00:07 [scrapy.addons] INFO: Enabled addons:
[]
2025-12-05 20:00:07 [asyncio] DEBUG: Using selector: EpollSelector
2025-12-05 20:00:07 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.asyncioreactor.AsyncioSelectorReactor
2025-12-05 20:00:07 [scrapy.utils.log] DEBUG: Using asyncio event loop: asyncio.unix_events._UnixSelectorEventLoop
2025-12-05 20:00:07 [scrapy.extensions.telnet] INFO: Telnet Password: c64c82007f983a19
2025-12-05 20:00:07 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.memusage.MemoryUsage',
 'scrapy.extensions.feedexport.FeedExporter',
 'scrapy.extensions.logstats.LogStats',
 'scrapy.extensions.closespider.CloseSpider']
2025-12-05 20:00:07 [scrapy.crawler] INFO: Overridden settings:
{'BOT_NAME': 'SourcingV2',
 'CLOSESPIDER_TIMEOUT': 7200,
 'FEED_EXPORT_ENCODING': 'utf-8',
 'LOG_FILE': '/var/log/scrapyd/logs/sourcing_v2/ayvens.es/task_54_2025-12-05T20_00_00.log',
 'LOG_FORMATTER': 'crawlers.log_formatter.SourcingLogFormatter',
 'NEWSPIDER_MODULE': 'spiders',
 'REQUEST_FINGERPRINTER_CLASS': 'scrapy_zyte_api.ScrapyZyteAPIRequestFingerprinter',
 'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7',
 'SPIDER_MODULES': ['spiders', 'auth_check'],
 'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor',
 'USER_AGENT': ''}
2025-12-05 20:00:07 [scrapy_zyte_api.handler] INFO: Using a Zyte API key starting with '5857011'
2025-12-05 20:00:07 [scrapy_zyte_api.handler] INFO: Using a Zyte API key starting with '5857011'
2025-12-05 20:00:07 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
 'scrapy_zyte_api.ScrapyZyteAPIDownloaderMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
 'scrapy.downloadermiddlewares.stats.DownloaderStats']
2025-12-05 20:00:07 [crawlers.middlewares.id_gen_middleware] INFO: Setting up IdGenerationMiddleware
2025-12-05 20:00:07 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
 'scrapy_zyte_api.ScrapyZyteAPISpiderMiddleware',
 'crawlers.middlewares.monitoring_spider_middleware.MonitoringSpiderMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware',
 'crawlers.middlewares.photo_download_middleware.PhotoDownloadMiddleware',
 'crawlers.middlewares.report_download_middleware.ReportDownloadMiddleware',
 'crawlers.middlewares.id_gen_middleware.IdGenMiddleware']
2025-12-05 20:00:07 [azure.identity._credentials.environment] INFO: Incomplete environment configuration for EnvironmentCredential. These variables are set: AZURE_CLIENT_ID, AZURE_TENANT_ID
2025-12-05 20:00:07 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:07 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:00:07 [azure.identity._credentials.chained] INFO: DefaultAzureCredential acquired a token from WorkloadIdentityCredential
2025-12-05 20:00:07 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxsourcingstorageprod.table.core.windows.net:443
2025-12-05 20:00:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:07 [crawlers.pipelines.translation_pipeline] INFO: Loading translations for language: en
2025-12-05 20:00:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /Translations()?$filter=PartitionKey%20eq%20%27en%27%20and%20RowKey%20eq%20%27en%27 HTTP/1.1" 200 None
2025-12-05 20:00:07 [crawlers.pipelines.item_rules_pipeline] INFO: Setting up ItemRules Pipeline
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: low_mileage_for_country.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: high_mileage.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: invalid_location_for_country.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: damaged_cars_from_auction_title.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: puretech_for_country.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: puretech_for_fr.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: missing_photos.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: damaged_from_info.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: makes_not_allowed.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: not_operable_from_info.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: cars_too_new_for_country.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: high_price_for_currency.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: makes_models_not_allowed.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: keywords_from_title.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: imported_cars.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: invalid_currency.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: high_mileage_for_country.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: missing_mileage.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: keywords_from_auction_title.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: invalid_country_of_origin.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: puretech_for_pt.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: electric_cars.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: registration_date_old.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: high_mileage_for_make.json
2025-12-05 20:00:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: missing_color.json
2025-12-05 20:00:07 [crawlers.pipelines.post_to_api] INFO: Setting up PostToApi Pipeline pointing to https://api.app.infinit.cc/api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing
2025-12-05 20:00:07 [scrapy.middleware] INFO: Enabled item pipelines:
['crawlers.pipelines.translation_pipeline.TranslationPipeline',
 'crawlers.pipelines.item_rules_pipeline.ItemRulesPipeline',
 'crawlers.pipelines.post_to_api.PostToApiPipeline']
2025-12-05 20:00:07 [scrapy.core.engine] INFO: Spider opened
2025-12-05 20:00:07 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2025-12-05 20:00:07 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2025-12-05 20:00:07 [scrapy-playwright] INFO: Starting download handler
2025-12-05 20:00:07 [scrapy-playwright] INFO: Starting download handler
2025-12-05 20:00:12 [scrapy-playwright] INFO: Launching browser firefox
2025-12-05 20:00:13 [scrapy-playwright] INFO: Browser firefox launched
2025-12-05 20:00:13 [scrapy-playwright] DEBUG: Browser context started: 'default' (persistent=False, remote=False)
2025-12-05 20:00:13 [scrapy-playwright] DEBUG: [Context=default] New page created, page count is 1 (1 for all contexts)
2025-12-05 20:00:13 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/en-gb/lots> (resource type: document)
2025-12-05 20:00:14 [scrapy-playwright] DEBUG: [Context=default] Response: <407 https://carmarket.ayvens.com/en-gb/lots>
2025-12-05 20:00:14 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/en-gb/lots> (resource type: document)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/en-gb/lots>
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/styles/themes/fonts/ald/Chillax-Medium.woff2> (resource type: font, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/styles/themes/fonts/ald/Chillax-Regular.woff2> (resource type: font, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/styles/themes/fonts/ald/Chillax-Semibold.woff2> (resource type: font, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/styles/themes/fonts/ald/SourceSans3-Bold.woff2> (resource type: font, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/styles/themes/fonts/ald/SourceSans3-Medium.woff2> (resource type: font, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/styles/themes/fonts/ald/SourceSans3-Regular.woff2> (resource type: font, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/styles/themes/fonts/ald/SourceSans3-SemiBold.woff2> (resource type: font, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-4QVFFAQG.js> (resource type: script, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-6BTLJK3G.js> (resource type: script, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-WFA2IVWG.js> (resource type: script, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-OF6JT4KC.js> (resource type: script, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-C7UPSRHK.js> (resource type: script, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-2N74OY7W.js> (resource type: script, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-X4BIG7ZC.js> (resource type: script, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-P3E7UHUI.js> (resource type: script, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-WE77U2QS.js> (resource type: script, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-2S63FZXS.js> (resource type: script, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/polyfills-FFHMD2TL.js> (resource type: script, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/main-LGMHLL25.js> (resource type: script, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/styles-ETA5EMQJ.css> (resource type: stylesheet, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/Chillax-Regular-IWU2LEHA.woff2> (resource type: font, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/Chillax-Medium-NIVAGDSI.woff2> (resource type: font, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/Chillax-Semibold-YNVWM5XA.woff2> (resource type: font, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/images/logo-ayvens-carmarket.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Aborted Playwright request <GET https://carmarket.ayvens.com/assets/images/logo-ayvens-carmarket.svg>
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/images/flags/round-gb.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Aborted Playwright request <GET https://carmarket.ayvens.com/assets/images/flags/round-gb.svg>
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/images/line.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Aborted Playwright request <GET https://carmarket.ayvens.com/assets/images/line.svg>
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/images/chevron-down.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Aborted Playwright request <GET https://carmarket.ayvens.com/assets/images/chevron-down.svg>
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/images/loader.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Aborted Playwright request <GET https://carmarket.ayvens.com/assets/images/loader.svg>
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/assets/styles/themes/fonts/ald/Chillax-Medium.woff2>
2025-12-05 20:00:16 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/assets/styles/themes/fonts/ald/Chillax-Regular.woff2>
2025-12-05 20:00:17 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/assets/styles/themes/fonts/ald/SourceSans3-Bold.woff2>
2025-12-05 20:00:18 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/assets/styles/themes/fonts/ald/Chillax-Semibold.woff2>
2025-12-05 20:00:18 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/assets/styles/themes/fonts/ald/SourceSans3-Medium.woff2>
2025-12-05 20:00:18 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-6BTLJK3G.js>
2025-12-05 20:00:18 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-PGNL7WWW.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-6BTLJK3G.js)
2025-12-05 20:00:18 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-RQLFCBMH.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-6BTLJK3G.js)
2025-12-05 20:00:18 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-IOF7VMBJ.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-6BTLJK3G.js)
2025-12-05 20:00:19 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-4CLCTAJ7.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-6BTLJK3G.js)
2025-12-05 20:00:19 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-4QVFFAQG.js>
2025-12-05 20:00:19 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-G7VKZLNV.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-4QVFFAQG.js)
2025-12-05 20:00:19 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/assets/styles/themes/fonts/ald/SourceSans3-Regular.woff2>
2025-12-05 20:00:19 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-WFA2IVWG.js>
2025-12-05 20:00:19 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-3HJTOVT5.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-WFA2IVWG.js)
2025-12-05 20:00:19 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/assets/styles/themes/fonts/ald/SourceSans3-SemiBold.woff2>
2025-12-05 20:00:20 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-OF6JT4KC.js>
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-C7UPSRHK.js>
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-2N74OY7W.js>
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-P3E7UHUI.js>
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-2S63FZXS.js>
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-AEILTBLO.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-C7UPSRHK.js)
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-HGXSF4AK.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-C7UPSRHK.js)
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-AQBI4CYB.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-C7UPSRHK.js)
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-X4BIG7ZC.js>
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-5HD3KQV3.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-X4BIG7ZC.js)
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-FB6RKR5I.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-X4BIG7ZC.js)
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-2BYH6G4C.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-X4BIG7ZC.js)
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-ALI5XYZJ.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-X4BIG7ZC.js)
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-VGUEXTON.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-X4BIG7ZC.js)
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-LHCNJE3I.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-X4BIG7ZC.js)
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-7DYAUIPQ.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-X4BIG7ZC.js)
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-7C2GTJ24.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-X4BIG7ZC.js)
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/polyfills-FFHMD2TL.js>
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/styles-ETA5EMQJ.css>
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/Chillax-Regular-IWU2LEHA.woff2> (resource type: font, referrer: https://carmarket.ayvens.com/styles-ETA5EMQJ.css)
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/Chillax-Medium-NIVAGDSI.woff2> (resource type: font, referrer: https://carmarket.ayvens.com/styles-ETA5EMQJ.css)
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/SourceSans3-Bold-OH3V3LEB.woff2> (resource type: font, referrer: https://carmarket.ayvens.com/styles-ETA5EMQJ.css)
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/SourceSans3-Regular-VMJBJIXS.woff2> (resource type: font, referrer: https://carmarket.ayvens.com/styles-ETA5EMQJ.css)
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/Chillax-Semibold-YNVWM5XA.woff2> (resource type: font, referrer: https://carmarket.ayvens.com/styles-ETA5EMQJ.css)
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/SourceSans3-SemiBold-NPKTOIJ5.woff2> (resource type: font, referrer: https://carmarket.ayvens.com/styles-ETA5EMQJ.css)
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/assets/Chillax-Medium-NIVAGDSI.woff2>
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-WE77U2QS.js>
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/assets/Chillax-Semibold-YNVWM5XA.woff2>
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/assets/Chillax-Regular-IWU2LEHA.woff2>
2025-12-05 20:00:21 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/main-LGMHLL25.js>
2025-12-05 20:00:22 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-PGNL7WWW.js>
2025-12-05 20:00:22 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-RQLFCBMH.js>
2025-12-05 20:00:22 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-CMYMW4RH.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:22 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-276SS4TV.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:22 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-J7KVSNWE.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:22 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-SPE4D2IX.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:22 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-OION63CA.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:22 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-NO5FJ6IE.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:22 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-AVZZH6AV.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:22 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-EAK24FOO.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:22 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-MADFXCQX.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:22 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-ABBXN5WQ.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-57LOL62V.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-MLMYZRAG.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-JK3VA6V3.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-ECZFIAAV.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-RWL6TUXQ.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-3ZRJENLY.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-AYT4KML6.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-35VRT2WM.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-CX65R3QQ.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-WJPXCSUZ.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-GWHLDUVT.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-LYOOO25D.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-7LLFTWQ2.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-XCHSKDZH.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-4CLCTAJ7.js>
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-IOF7VMBJ.js>
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-G7VKZLNV.js>
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-3HJTOVT5.js>
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-AEILTBLO.js>
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-HGXSF4AK.js>
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-AQBI4CYB.js>
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-5HD3KQV3.js>
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-FB6RKR5I.js>
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-2BYH6G4C.js>
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-ALI5XYZJ.js>
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-VGUEXTON.js>
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-LHCNJE3I.js>
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-7DYAUIPQ.js>
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-7C2GTJ24.js>
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/assets/Chillax-Regular-IWU2LEHA.woff2>
2025-12-05 20:00:23 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/assets/SourceSans3-SemiBold-NPKTOIJ5.woff2>
2025-12-05 20:00:24 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/assets/Chillax-Medium-NIVAGDSI.woff2>
2025-12-05 20:00:24 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/assets/SourceSans3-Bold-OH3V3LEB.woff2>
2025-12-05 20:00:24 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/assets/SourceSans3-Regular-VMJBJIXS.woff2>
2025-12-05 20:00:24 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/assets/Chillax-Semibold-YNVWM5XA.woff2>
2025-12-05 20:00:25 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-CMYMW4RH.js>
2025-12-05 20:00:25 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-276SS4TV.js>
2025-12-05 20:00:25 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-J7KVSNWE.js>
2025-12-05 20:00:25 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-SPE4D2IX.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-OION63CA.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-NO5FJ6IE.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-AVZZH6AV.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-EAK24FOO.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-MADFXCQX.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-ABBXN5WQ.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-57LOL62V.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-MLMYZRAG.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-JK3VA6V3.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-ECZFIAAV.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-RWL6TUXQ.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-3ZRJENLY.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-AYT4KML6.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-35VRT2WM.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-CX65R3QQ.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-WJPXCSUZ.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-GWHLDUVT.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-LYOOO25D.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-7LLFTWQ2.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-XCHSKDZH.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://js.monitor.azure.com/scripts/b/ai.config.1.cfg.json> (resource type: fetch, referrer: https://carmarket.ayvens.com/)
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-U3DCL5KE.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-7L4G66MJ.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/images/favicon/ayvens-white-bg_192x192.png> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Aborted Playwright request <GET https://carmarket.ayvens.com/assets/images/favicon/ayvens-white-bg_192x192.png>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/images/favicon/ayvens.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Aborted Playwright request <GET https://carmarket.ayvens.com/assets/images/favicon/ayvens.svg>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-U3DCL5KE.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-7L4G66MJ.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-LOIZFXZY.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-7L4G66MJ.js)
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-LOIZFXZY.js>
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-7P3LAPEA.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-LOIZFXZY.js)
2025-12-05 20:00:26 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://js.monitor.azure.com/scripts/b/ai.config.1.cfg.json>
2025-12-05 20:00:27 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-7P3LAPEA.js>
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-GQHESYQG.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-TXAFC7UM.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-3OV7MDAK.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-LOIZFXZY.js)
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-GQHESYQG.js>
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-RJDFUE4N.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-GQHESYQG.js)
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-HPJAYUCB.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-GQHESYQG.js)
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-CJQ6ZMFR.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-GQHESYQG.js)
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-3OV7MDAK.js>
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-TXAFC7UM.js>
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-OIQIERBW.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-3OV7MDAK.js)
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-SOPV2ZIG.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-3OV7MDAK.js)
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-NTG4JAHX.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-3OV7MDAK.js)
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-7TECJER3.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-3OV7MDAK.js)
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-H7VMIGZK.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-3OV7MDAK.js)
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-RJDFUE4N.js>
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-HPJAYUCB.js>
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-CJQ6ZMFR.js>
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-OIQIERBW.js>
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-NTG4JAHX.js>
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-7TECJER3.js>
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-SOPV2ZIG.js>
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-H7VMIGZK.js>
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/images/loader.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:28 [scrapy-playwright] DEBUG: [Context=default] Aborted Playwright request <GET https://carmarket.ayvens.com/assets/images/loader.svg>
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/images/flags/round-gb.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Aborted Playwright request <GET https://carmarket.ayvens.com/assets/images/flags/round-gb.svg>
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/images/line.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Aborted Playwright request <GET https://carmarket.ayvens.com/assets/images/line.svg>
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/images/chevron-down.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Aborted Playwright request <GET https://carmarket.ayvens.com/assets/images/chevron-down.svg>
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/images/logo-ayvens-carmarket.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Aborted Playwright request <GET https://carmarket.ayvens.com/assets/images/logo-ayvens-carmarket.svg>
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <POST https://dc.services.visualstudio.com/v2/track> (resource type: xhr, referrer: https://carmarket.ayvens.com/)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-K3XLXP2R.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-3OV7MDAK.js)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-JGZVQ3UU.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-SOPV2ZIG.js)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-HXFSIYAP.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-TXAFC7UM.js)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-QMSFYIWN.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-TXAFC7UM.js)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <POST https://dc.services.visualstudio.com/v2/track> (resource type: xhr, referrer: https://carmarket.ayvens.com/)
2025-12-05 20:00:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://carmarket.ayvens.com/en-gb/lots> (referer: None) ['playwright']
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <POST https://dc.services.visualstudio.com/v2/track> (resource type: xhr, referrer: https://carmarket.ayvens.com/)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-K3XLXP2R.js>
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/images/down.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Aborted Playwright request <GET https://carmarket.ayvens.com/assets/images/down.svg>
2025-12-05 20:00:29 [ayvens.es] INFO: Cookies from Playwright: [{'name': 'ai_user', 'value': 'v6uOabosC5mJH51geyxd/0|2025-12-05T20:00:26.119Z', 'domain': 'carmarket.ayvens.com', 'path': '/', 'expires': 1796500826, 'httpOnly': False, 'secure': True, 'sameSite': 'None'}, {'name': 'user_culture', 'value': 'en-gb', 'domain': 'carmarket.ayvens.com', 'path': '/', 'expires': -1, 'httpOnly': False, 'secure': False, 'sameSite': 'None'}, {'name': 'ai_session', 'value': 'Bgdvlxvy4tQ8DZklrTy1CY|1764964828875|1764964828875', 'domain': 'carmarket.ayvens.com', 'path': '/', 'expires': 1764966628, 'httpOnly': False, 'secure': True, 'sameSite': 'None'}]
2025-12-05 20:00:29 [ayvens.es] INFO: Scrapy-formatted cookies: [{'name': 'ai_user', 'value': 'v6uOabosC5mJH51geyxd/0|2025-12-05T20:00:26.119Z', 'domain': 'carmarket.ayvens.com', 'path': '/'}, {'name': 'user_culture', 'value': 'en-gb', 'domain': 'carmarket.ayvens.com', 'path': '/'}, {'name': 'ai_session', 'value': 'Bgdvlxvy4tQ8DZklrTy1CY|1764964828875|1764964828875', 'domain': 'carmarket.ayvens.com', 'path': '/'}]
2025-12-05 20:00:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /SpiderData(PartitionKey='Ayvens',RowKey='cookies') HTTP/1.1" 204 0
2025-12-05 20:00:29 [ayvens.es] INFO: Intercepting token...
2025-12-05 20:00:29 [ayvens.es] INFO: Scrolling element to trigger lazy load...
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-JGZVQ3UU.js>
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-WUZ7BWR5.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-JGZVQ3UU.js)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-LQW6USBM.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-JGZVQ3UU.js)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-HXFSIYAP.js>
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-QMSFYIWN.js>
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/images/bell.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:29 [ayvens.es] INFO: GraphQL headers: {'host': 'api.aldcarmarket.com', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:125.0) Gecko/20100101 Firefox/125.0', 'accept': 'application/json, text/plain, */*', 'accept-language': 'en-US,en;q=0.5', 'accept-encoding': 'gzip, deflate, br', 'content-type': 'application/json', 'content-length': '839', 'referer': 'https://carmarket.ayvens.com/', 'x-ald-subscription-key': '3b2cc62fd26c4e29a762db3de181266b', 'x-tenant': 'ald', 'x-country': 'gb', 'origin': 'https://carmarket.ayvens.com', 'sec-fetch-dest': 'empty', 'sec-fetch-mode': 'cors', 'sec-fetch-site': 'cross-site', 'proxy-authorization': 'Basic Y3VzdG9tZXItaW5maW5pdF9zb3VyY2luZ192Ml9QdGpxbi1jYy1FVTpLSks1eEVqSGtGek1GOFRBWEJoek1NckV5dFVYamZ1ZnhYQ0YzOFQ3ejMwNGo5TmtqRDJGNnhTdzdqNGRBdGFE', 'connection': 'keep-alive'}
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/images/next-breadcrumb.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-DEAFIDEM.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <POST https://api.aldcarmarket.com/graphql/saleevents> (resource type: xhr, referrer: https://carmarket.ayvens.com/)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/images/logo-ayvens.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/en-gb/content/trading/-/media/Project/ALD/shared/flags/Country-Sales/flag-fr.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/en-gb/content/trading/-/media/Project/ALD/shared/flags/Country-Sales/flag-es.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/en-gb/content/trading/-/media/Project/ALD/shared/flags/Country-Sales/flag-be.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/en-gb/content/trading/-/media/Project/ALD/shared/flags/Country-Sales/flag-de.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/en-gb/content/trading/-/media/Project/ALD/shared/flags/Country-Sales/flag-it.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/images/loader.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-WUZ7BWR5.js>
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/images/chevron-down.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-LQW6USBM.js>
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/assets/images/bell.svg>
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/assets/images/next-breadcrumb.svg>
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-DEAFIDEM.js>
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/assets/images/logo-ayvens.svg>
2025-12-05 20:00:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-C5U7N2E6.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-DEAFIDEM.js)
2025-12-05 20:00:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/en-gb/content/trading/-/media/Project/ALD/shared/flags/Country-Sales/flag-fr.svg>
2025-12-05 20:00:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/assets/images/loader.svg>
2025-12-05 20:00:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/en-gb/content/trading/-/media/Project/ALD/shared/flags/Country-Sales/flag-de.svg>
2025-12-05 20:00:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/assets/images/chevron-down.svg>
2025-12-05 20:00:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/en-gb/content/trading/-/media/Project/ALD/shared/flags/Country-Sales/flag-it.svg>
2025-12-05 20:00:30 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-6YCHXE2J.js> (resource type: script, referrer: https://carmarket.ayvens.com/main-LGMHLL25.js)
2025-12-05 20:00:30 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/images/down.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:30 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/assets/images/flags/be.svg> (resource type: image, referrer: https://carmarket.ayvens.com/en-gb/lots)
2025-12-05 20:00:30 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://carmarket.ayvens.com/chunk-LMJCIPKN.js> (resource type: script, referrer: https://carmarket.ayvens.com/chunk-7TECJER3.js)
2025-12-05 20:00:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://dc.services.visualstudio.com/v2/track>
2025-12-05 20:00:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://dc.services.visualstudio.com/v2/track>
2025-12-05 20:00:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-C5U7N2E6.js>
2025-12-05 20:00:30 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://cdn.cookielaw.org/scripttemplates/otSDKStub.js> (resource type: script, referrer: https://carmarket.ayvens.com/)
2025-12-05 20:00:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://dc.services.visualstudio.com/v2/track>
2025-12-05 20:00:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-6YCHXE2J.js>
2025-12-05 20:00:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/assets/images/down.svg>
2025-12-05 20:00:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/assets/images/flags/be.svg>
2025-12-05 20:00:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://carmarket.ayvens.com/chunk-LMJCIPKN.js>
2025-12-05 20:00:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://cdn.cookielaw.org/scripttemplates/otSDKStub.js>
2025-12-05 20:00:30 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://cdn.cookielaw.org/consent/019908df-58e4-7710-aaa7-4d6e79279bd9/019908df-58e4-7710-aaa7-4d6e79279bd9.json> (resource type: xhr, referrer: https://carmarket.ayvens.com/)
2025-12-05 20:00:31 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://cdn.cookielaw.org/consent/019908df-58e4-7710-aaa7-4d6e79279bd9/019908df-58e4-7710-aaa7-4d6e79279bd9.json>
2025-12-05 20:00:31 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://cdn.cookielaw.org/scripttemplates/202509.1.0/otBannerSdk.js> (resource type: script, referrer: https://carmarket.ayvens.com/)
2025-12-05 20:00:31 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://cdn.cookielaw.org/scripttemplates/202509.1.0/otBannerSdk.js>
2025-12-05 20:00:31 [ayvens.es] INFO: ✅ Intercepted token: 3b2cc62fd26c4e29a762db3de181266b
2025-12-05 20:00:31 [ayvens.es] INFO: Fetching listings with token: 3b2cc62fd26c4e29a762db3de181266b and skip: 0
2025-12-05 20:00:31 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 1st time calling it.
2025-12-05 20:00:33 [scrapy.core.engine] DEBUG: Crawled (200) <POST https://api.aldcarmarket.com/graphql/saleevents> (referer: https://carmarket.ayvens.com/) ['zyte-api']
2025-12-05 20:00:33 [ayvens.es] INFO: Found listing with ID: 1788736
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788736%27 HTTP/1.1" 200 None
2025-12-05 20:00:33 [ayvens.es] INFO: Scrape type for 1788736: 1
2025-12-05 20:00:33 [ayvens.es] INFO: Extracting data for car ID: 1788736
2025-12-05 20:00:33 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: d82f5644-1847-5752-91f1-afeccfa71fb1 with identifier: 1788736
2025-12-05 20:00:33 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item d82f5644-1847-5752-91f1-afeccfa71fb1 with scrape type 1
2025-12-05 20:00:33 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item d82f5644-1847-5752-91f1-afeccfa71fb1
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F075%252Fb2686d0d08fc4512b0b8224363447e5e%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/075/b2686d0d08fc4512b0b8224363447e5e/images/large/1.jpg not downloaded yet
2025-12-05 20:00:33 [ayvens.es] INFO: Found listing with ID: 1788737
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788737%27 HTTP/1.1" 200 None
2025-12-05 20:00:33 [ayvens.es] INFO: Scrape type for 1788737: 1
2025-12-05 20:00:33 [ayvens.es] INFO: Extracting data for car ID: 1788737
2025-12-05 20:00:33 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 323c7d47-482a-5e80-8fad-70199a8a392b with identifier: 1788737
2025-12-05 20:00:33 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 323c7d47-482a-5e80-8fad-70199a8a392b with scrape type 1
2025-12-05 20:00:33 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 323c7d47-482a-5e80-8fad-70199a8a392b
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F580%252F5ebf9b678fce465f9879db0b6783cf91%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/580/5ebf9b678fce465f9879db0b6783cf91/images/large/1.jpg not downloaded yet
2025-12-05 20:00:33 [ayvens.es] INFO: Found listing with ID: 1788738
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788738%27 HTTP/1.1" 200 None
2025-12-05 20:00:33 [ayvens.es] INFO: Scrape type for 1788738: 1
2025-12-05 20:00:33 [ayvens.es] INFO: Extracting data for car ID: 1788738
2025-12-05 20:00:33 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: bf858f6a-7a47-53fd-9bc6-83336e169fe9 with identifier: 1788738
2025-12-05 20:00:33 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item bf858f6a-7a47-53fd-9bc6-83336e169fe9 with scrape type 1
2025-12-05 20:00:33 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item bf858f6a-7a47-53fd-9bc6-83336e169fe9
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F633%252F31ccd8adad4c473b89f9d42ab3ed9978%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/633/31ccd8adad4c473b89f9d42ab3ed9978/images/large/1.jpg not downloaded yet
2025-12-05 20:00:33 [ayvens.es] INFO: Found listing with ID: 1788739
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788739%27 HTTP/1.1" 200 None
2025-12-05 20:00:33 [ayvens.es] INFO: Scrape type for 1788739: 1
2025-12-05 20:00:33 [ayvens.es] INFO: Extracting data for car ID: 1788739
2025-12-05 20:00:33 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: dd9b4173-cdcc-5e6c-b346-05f9e1df0704 with identifier: 1788739
2025-12-05 20:00:33 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item dd9b4173-cdcc-5e6c-b346-05f9e1df0704 with scrape type 1
2025-12-05 20:00:33 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item dd9b4173-cdcc-5e6c-b346-05f9e1df0704
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F634%252F338a9c6cab0347b28f72ce5ef6bf7eec%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/634/338a9c6cab0347b28f72ce5ef6bf7eec/images/large/1.jpg not downloaded yet
2025-12-05 20:00:33 [ayvens.es] INFO: Found listing with ID: 1788740
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788740%27 HTTP/1.1" 200 None
2025-12-05 20:00:33 [ayvens.es] INFO: Scrape type for 1788740: 1
2025-12-05 20:00:33 [ayvens.es] INFO: Extracting data for car ID: 1788740
2025-12-05 20:00:33 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 879c64c5-9a6c-53c0-a91e-a73a4704d491 with identifier: 1788740
2025-12-05 20:00:33 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 879c64c5-9a6c-53c0-a91e-a73a4704d491 with scrape type 1
2025-12-05 20:00:33 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 879c64c5-9a6c-53c0-a91e-a73a4704d491
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F788%252F0024ef09da274917a7c417d43415be1f%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/788/0024ef09da274917a7c417d43415be1f/images/large/1.jpg not downloaded yet
2025-12-05 20:00:33 [ayvens.es] INFO: Found listing with ID: 1788741
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788741%27 HTTP/1.1" 200 None
2025-12-05 20:00:33 [ayvens.es] INFO: Scrape type for 1788741: 1
2025-12-05 20:00:33 [ayvens.es] INFO: Extracting data for car ID: 1788741
2025-12-05 20:00:33 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: b7c086f0-43e5-5cb6-898a-38aef0e17447 with identifier: 1788741
2025-12-05 20:00:33 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b7c086f0-43e5-5cb6-898a-38aef0e17447 with scrape type 1
2025-12-05 20:00:33 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item b7c086f0-43e5-5cb6-898a-38aef0e17447
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F327%252F5f42d39bfd2f4da4984101230f581da9%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/327/5f42d39bfd2f4da4984101230f581da9/images/large/1.jpg not downloaded yet
2025-12-05 20:00:33 [ayvens.es] INFO: Found listing with ID: 1788742
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788742%27 HTTP/1.1" 200 None
2025-12-05 20:00:33 [ayvens.es] INFO: Scrape type for 1788742: 1
2025-12-05 20:00:33 [ayvens.es] INFO: Extracting data for car ID: 1788742
2025-12-05 20:00:33 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 79911923-59d9-542a-be1f-742ee1ad6b22 with identifier: 1788742
2025-12-05 20:00:33 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 79911923-59d9-542a-be1f-742ee1ad6b22 with scrape type 1
2025-12-05 20:00:33 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 79911923-59d9-542a-be1f-742ee1ad6b22
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F980%252Fa1224eea2b5f405d893728d9425d0e3f%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/980/a1224eea2b5f405d893728d9425d0e3f/images/large/1.jpg not downloaded yet
2025-12-05 20:00:33 [ayvens.es] INFO: Found listing with ID: 1788743
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788743%27 HTTP/1.1" 200 None
2025-12-05 20:00:33 [ayvens.es] INFO: Scrape type for 1788743: 1
2025-12-05 20:00:33 [ayvens.es] INFO: Extracting data for car ID: 1788743
2025-12-05 20:00:33 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: a3c678ce-869c-5147-a58e-bb4699807d0b with identifier: 1788743
2025-12-05 20:00:33 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a3c678ce-869c-5147-a58e-bb4699807d0b with scrape type 1
2025-12-05 20:00:33 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item a3c678ce-869c-5147-a58e-bb4699807d0b
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F907%252F443211875eaa4e5eb6f7ebdff333ef46%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/907/443211875eaa4e5eb6f7ebdff333ef46/images/large/1.jpg not downloaded yet
2025-12-05 20:00:33 [ayvens.es] INFO: Found listing with ID: 1788744
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788744%27 HTTP/1.1" 200 None
2025-12-05 20:00:33 [ayvens.es] INFO: Scrape type for 1788744: 1
2025-12-05 20:00:33 [ayvens.es] INFO: Extracting data for car ID: 1788744
2025-12-05 20:00:33 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 43c66767-66a7-5d52-91ef-7193825793ca with identifier: 1788744
2025-12-05 20:00:33 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 43c66767-66a7-5d52-91ef-7193825793ca with scrape type 1
2025-12-05 20:00:33 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 43c66767-66a7-5d52-91ef-7193825793ca
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F331%252F30582bd8c7c64080a21e0c55a2f0181d%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/331/30582bd8c7c64080a21e0c55a2f0181d/images/large/1.jpg not downloaded yet
2025-12-05 20:00:33 [ayvens.es] INFO: Found listing with ID: 1788745
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788745%27 HTTP/1.1" 200 None
2025-12-05 20:00:33 [ayvens.es] INFO: Scrape type for 1788745: 1
2025-12-05 20:00:33 [ayvens.es] INFO: Extracting data for car ID: 1788745
2025-12-05 20:00:33 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: b9072299-b299-5048-bc40-359724dfebbc with identifier: 1788745
2025-12-05 20:00:33 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b9072299-b299-5048-bc40-359724dfebbc with scrape type 1
2025-12-05 20:00:33 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item b9072299-b299-5048-bc40-359724dfebbc
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F398%252F23ecebe217c24c009ace313e6cd39319%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/398/23ecebe217c24c009ace313e6cd39319/images/large/1.jpg not downloaded yet
2025-12-05 20:00:33 [ayvens.es] INFO: Found listing with ID: 1788746
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788746%27 HTTP/1.1" 200 None
2025-12-05 20:00:33 [ayvens.es] INFO: Scrape type for 1788746: 1
2025-12-05 20:00:33 [ayvens.es] INFO: Extracting data for car ID: 1788746
2025-12-05 20:00:33 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 487d4f7c-483b-5c49-809c-258c5f533c19 with identifier: 1788746
2025-12-05 20:00:33 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 487d4f7c-483b-5c49-809c-258c5f533c19 with scrape type 1
2025-12-05 20:00:33 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 0 photos for item 487d4f7c-483b-5c49-809c-258c5f533c19
2025-12-05 20:00:33 [crawlers.middlewares.photo_download_middleware] INFO: No photos to download for item 487d4f7c-483b-5c49-809c-258c5f533c19
2025-12-05 20:00:33 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 487d4f7c-483b-5c49-809c-258c5f533c19, identifier: 1788746
2025-12-05 20:00:33 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 487d4f7c-483b-5c49-809c-258c5f533c19 with identifier: 1788746
2025-12-05 20:00:33 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:00:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:00:33 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:00:34 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 487d4f7c-483b-5c49-809c-258c5f533c19 with identifier: 1788746
2025-12-05 20:00:34 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'renault' containing 'renault' and model 'kangoo express' containing 'express'
2025-12-05 20:00:34 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 487d4f7c-483b-5c49-809c-258c5f533c19 spider: ayvens.es
2025-12-05 20:00:34 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 487d4f7c-483b-5c49-809c-258c5f533c19 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'renault' containing 'renault' and model 'kangoo express' containing 'express'
{'auction_title': '',
 'c_o_2_emission_value': 117,
 'color': 'Pintura',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '487d4f7c-483b-5c49-809c-258c5f533c19',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'KANGOO EXPRESS',
 'odometer': 31549,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 0,
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-07-09',
 'seats_number': 2,
 'seller_name': None,
 'title': 'RENAULT KANGOO EXPRESS Maxi 2p Blue dCi 70 kW (95CV)',
 'transmission': 'manual',
 'trim': 'Maxi 2p Blue dCi 70 kW (95CV)',
 'vin': None,
 'year': 2020}
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788746%27 HTTP/1.1" 200 None
2025-12-05 20:00:34 [ayvens.es] INFO: Saving data for 1788746: {'created_time': 1764964834.243087, 'last_price_update_time': 1764964834.243103, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788746') HTTP/1.1" 204 0
2025-12-05 20:00:34 [ayvens.es] INFO: Found listing with ID: 1788747
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788747%27 HTTP/1.1" 200 None
2025-12-05 20:00:34 [ayvens.es] INFO: Scrape type for 1788747: 1
2025-12-05 20:00:34 [ayvens.es] INFO: Extracting data for car ID: 1788747
2025-12-05 20:00:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 34b30405-e26c-57ed-8abe-3601873e87d8 with identifier: 1788747
2025-12-05 20:00:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 34b30405-e26c-57ed-8abe-3601873e87d8 with scrape type 1
2025-12-05 20:00:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 34b30405-e26c-57ed-8abe-3601873e87d8
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F893%252Fb6d4c28d896d4f5da7ff1fab4a9b33b2%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/893/b6d4c28d896d4f5da7ff1fab4a9b33b2/images/large/1.jpg not downloaded yet
2025-12-05 20:00:34 [ayvens.es] INFO: Found listing with ID: 1788748
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788748%27 HTTP/1.1" 200 None
2025-12-05 20:00:34 [ayvens.es] INFO: Scrape type for 1788748: 1
2025-12-05 20:00:34 [ayvens.es] INFO: Extracting data for car ID: 1788748
2025-12-05 20:00:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: ba23f778-25cd-5eb2-9d2d-47c45a866f9f with identifier: 1788748
2025-12-05 20:00:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ba23f778-25cd-5eb2-9d2d-47c45a866f9f with scrape type 1
2025-12-05 20:00:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item ba23f778-25cd-5eb2-9d2d-47c45a866f9f
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F319%252F0c141d56ad1a4ba5aa8049bfc73e0f2c%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/319/0c141d56ad1a4ba5aa8049bfc73e0f2c/images/large/1.jpg not downloaded yet
2025-12-05 20:00:34 [ayvens.es] INFO: Found listing with ID: 1788749
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788749%27 HTTP/1.1" 200 None
2025-12-05 20:00:34 [ayvens.es] INFO: Scrape type for 1788749: 1
2025-12-05 20:00:34 [ayvens.es] INFO: Extracting data for car ID: 1788749
2025-12-05 20:00:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 116c9546-b2b2-5b81-a633-fe0f0e82c12a with identifier: 1788749
2025-12-05 20:00:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 116c9546-b2b2-5b81-a633-fe0f0e82c12a with scrape type 1
2025-12-05 20:00:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 116c9546-b2b2-5b81-a633-fe0f0e82c12a
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F944%252F47b913861add4db28da3355976d19c19%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/944/47b913861add4db28da3355976d19c19/images/large/1.jpg not downloaded yet
2025-12-05 20:00:34 [ayvens.es] INFO: Found listing with ID: 1788750
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788750%27 HTTP/1.1" 200 None
2025-12-05 20:00:34 [ayvens.es] INFO: Scrape type for 1788750: 1
2025-12-05 20:00:34 [ayvens.es] INFO: Extracting data for car ID: 1788750
2025-12-05 20:00:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 9846a71a-949c-5f22-98a6-75322fa13620 with identifier: 1788750
2025-12-05 20:00:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9846a71a-949c-5f22-98a6-75322fa13620 with scrape type 1
2025-12-05 20:00:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 9846a71a-949c-5f22-98a6-75322fa13620
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F575%252Faddcb1c702614f949522fc5c38893a15%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/575/addcb1c702614f949522fc5c38893a15/images/large/1.jpg not downloaded yet
2025-12-05 20:00:34 [ayvens.es] INFO: Found listing with ID: 1788751
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788751%27 HTTP/1.1" 200 None
2025-12-05 20:00:34 [ayvens.es] INFO: Scrape type for 1788751: 1
2025-12-05 20:00:34 [ayvens.es] INFO: Extracting data for car ID: 1788751
2025-12-05 20:00:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: e957512c-cc59-50fd-b9c0-8801484aa268 with identifier: 1788751
2025-12-05 20:00:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e957512c-cc59-50fd-b9c0-8801484aa268 with scrape type 1
2025-12-05 20:00:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item e957512c-cc59-50fd-b9c0-8801484aa268
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F575%252F4e6c70c31cb044f584e682d6a6c7d864%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/575/4e6c70c31cb044f584e682d6a6c7d864/images/large/1.jpg not downloaded yet
2025-12-05 20:00:34 [ayvens.es] INFO: Found listing with ID: 1788752
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788752%27 HTTP/1.1" 200 None
2025-12-05 20:00:34 [ayvens.es] INFO: Scrape type for 1788752: 1
2025-12-05 20:00:34 [ayvens.es] INFO: Extracting data for car ID: 1788752
2025-12-05 20:00:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 6e43078d-5ac1-5dd1-b035-99747b2d3142 with identifier: 1788752
2025-12-05 20:00:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6e43078d-5ac1-5dd1-b035-99747b2d3142 with scrape type 1
2025-12-05 20:00:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 6e43078d-5ac1-5dd1-b035-99747b2d3142
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F866%252Fadae37cfb08344809baf7cf88779f315%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/866/adae37cfb08344809baf7cf88779f315/images/large/1.jpg not downloaded yet
2025-12-05 20:00:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/634/338a9c6cab0347b28f72ce5ef6bf7eec/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/633/31ccd8adad4c473b89f9d42ab3ed9978/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:34 [ayvens.es] INFO: Found listing with ID: 1788753
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788753%27 HTTP/1.1" 200 None
2025-12-05 20:00:34 [ayvens.es] INFO: Scrape type for 1788753: 1
2025-12-05 20:00:34 [ayvens.es] INFO: Extracting data for car ID: 1788753
2025-12-05 20:00:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 90be6337-5eee-580b-9438-6b5ee5cce03d with identifier: 1788753
2025-12-05 20:00:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 90be6337-5eee-580b-9438-6b5ee5cce03d with scrape type 1
2025-12-05 20:00:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 90be6337-5eee-580b-9438-6b5ee5cce03d
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F263%252F6fbfe82b8c2b41058b1e7ac6d094e687%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/263/6fbfe82b8c2b41058b1e7ac6d094e687/images/large/1.jpg not downloaded yet
2025-12-05 20:00:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/075/b2686d0d08fc4512b0b8224363447e5e/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:34 [ayvens.es] INFO: Found listing with ID: 1788754
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788754%27 HTTP/1.1" 200 None
2025-12-05 20:00:34 [ayvens.es] INFO: Scrape type for 1788754: 1
2025-12-05 20:00:34 [ayvens.es] INFO: Extracting data for car ID: 1788754
2025-12-05 20:00:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: f1c1361e-8b28-54cc-872c-4a7549e871af with identifier: 1788754
2025-12-05 20:00:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f1c1361e-8b28-54cc-872c-4a7549e871af with scrape type 1
2025-12-05 20:00:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item f1c1361e-8b28-54cc-872c-4a7549e871af
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F119%252F5726d2f38e1c491293a7f63dc64d5950%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/119/5726d2f38e1c491293a7f63dc64d5950/images/large/1.jpg not downloaded yet
2025-12-05 20:00:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/580/5ebf9b678fce465f9879db0b6783cf91/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/788/0024ef09da274917a7c417d43415be1f/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:34 [ayvens.es] INFO: Found listing with ID: 1788755
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788755%27 HTTP/1.1" 200 None
2025-12-05 20:00:35 [ayvens.es] INFO: Scrape type for 1788755: 1
2025-12-05 20:00:35 [ayvens.es] INFO: Extracting data for car ID: 1788755
2025-12-05 20:00:35 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: b7bda7b1-f7a2-568a-b818-31b0ac309fc1 with identifier: 1788755
2025-12-05 20:00:35 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b7bda7b1-f7a2-568a-b818-31b0ac309fc1 with scrape type 1
2025-12-05 20:00:35 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item b7bda7b1-f7a2-568a-b818-31b0ac309fc1
2025-12-05 20:00:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F591%252F55e129043e144cb3abf69e0b9d14c367%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/591/55e129043e144cb3abf69e0b9d14c367/images/large/1.jpg not downloaded yet
2025-12-05 20:00:35 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:35 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:35 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:35 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b289ff90>
2025-12-05 20:00:35 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:35 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:35 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:00:35 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:35 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:35 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:35 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_dd9b4173-cdcc-5e6c-b346-05f9e1df0704/dd9b4173-cdcc-5e6c-b346-05f9e1df0704_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F634%252F338a9c6cab0347b28f72ce5ef6bf7eec%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/634/338a9c6cab0347b28f72ce5ef6bf7eec/images/large/1.jpg saved to cache
2025-12-05 20:00:35 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item dd9b4173-cdcc-5e6c-b346-05f9e1df0704
2025-12-05 20:00:35 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: dd9b4173-cdcc-5e6c-b346-05f9e1df0704, skipping ID generation
2025-12-05 20:00:35 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item dd9b4173-cdcc-5e6c-b346-05f9e1df0704 with scrape type 1
2025-12-05 20:00:35 [crawlers.middlewares.photo_download_middleware] INFO: Removing item dd9b4173-cdcc-5e6c-b346-05f9e1df0704 from tracking dictionaries
2025-12-05 20:00:35 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item dd9b4173-cdcc-5e6c-b346-05f9e1df0704 sending to next pipeline
2025-12-05 20:00:35 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: dd9b4173-cdcc-5e6c-b346-05f9e1df0704, identifier: 1788739
2025-12-05 20:00:35 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:35 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:35 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:35 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b2714950>
2025-12-05 20:00:35 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:35 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:35 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:00:35 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:36 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:36 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:36 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_bf858f6a-7a47-53fd-9bc6-83336e169fe9/bf858f6a-7a47-53fd-9bc6-83336e169fe9_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F633%252F31ccd8adad4c473b89f9d42ab3ed9978%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:36 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/633/31ccd8adad4c473b89f9d42ab3ed9978/images/large/1.jpg saved to cache
2025-12-05 20:00:36 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item bf858f6a-7a47-53fd-9bc6-83336e169fe9
2025-12-05 20:00:36 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: bf858f6a-7a47-53fd-9bc6-83336e169fe9, skipping ID generation
2025-12-05 20:00:36 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item bf858f6a-7a47-53fd-9bc6-83336e169fe9 with scrape type 1
2025-12-05 20:00:36 [crawlers.middlewares.photo_download_middleware] INFO: Removing item bf858f6a-7a47-53fd-9bc6-83336e169fe9 from tracking dictionaries
2025-12-05 20:00:36 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item bf858f6a-7a47-53fd-9bc6-83336e169fe9 sending to next pipeline
2025-12-05 20:00:36 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: bf858f6a-7a47-53fd-9bc6-83336e169fe9, identifier: 1788738
2025-12-05 20:00:36 [ayvens.es] INFO: Found listing with ID: 1788756
2025-12-05 20:00:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788756%27 HTTP/1.1" 200 None
2025-12-05 20:00:36 [ayvens.es] INFO: Scrape type for 1788756: 1
2025-12-05 20:00:36 [ayvens.es] INFO: Extracting data for car ID: 1788756
2025-12-05 20:00:36 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 786d7a25-8513-559f-bf2d-612205bd215c with identifier: 1788756
2025-12-05 20:00:36 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 786d7a25-8513-559f-bf2d-612205bd215c with scrape type 1
2025-12-05 20:00:36 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 786d7a25-8513-559f-bf2d-612205bd215c
2025-12-05 20:00:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F809%252F18e3a6f9aaf64244b3cae0ff5e320441%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:36 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/809/18e3a6f9aaf64244b3cae0ff5e320441/images/large/1.jpg not downloaded yet
2025-12-05 20:00:36 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: dd9b4173-cdcc-5e6c-b346-05f9e1df0704 with identifier: 1788739
2025-12-05 20:00:36 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:00:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:00:36 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:00:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:00:36 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: dd9b4173-cdcc-5e6c-b346-05f9e1df0704 with identifier: 1788739
2025-12-05 20:00:36 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: dd9b4173-cdcc-5e6c-b346-05f9e1df0704 with identifier: 1788739 to the API
2025-12-05 20:00:36 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:00:36 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:00:36 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/634/338a9c6cab0347b28f72ce5ef6bf7eec/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 96,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'gas',
 'id': 'dd9b4173-cdcc-5e6c-b346-05f9e1df0704',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SEAT',
 'model': 'LEON',
 'odometer': 84659,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_dd9b4173-cdcc-5e6c-b346-05f9e1df0704/dd9b4173-cdcc-5e6c-b346-05f9e1df0704_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 356784,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/634/338a9c6cab0347b28f72ce5ef6bf7eec/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-08-06',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SEAT LEON ST 1.5 TGI 96kW DSG-7 S&S Xcellence',
 'transmission': 'manual',
 'trim': 'ST 1.5 TGI 96kW DSG-7 S&S Xcellence',
 'vin': None,
 'year': 2020}
2025-12-05 20:00:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788739%27 HTTP/1.1" 200 None
2025-12-05 20:00:36 [ayvens.es] INFO: Saving data for 1788739: {'created_time': 1764964836.591836, 'last_price_update_time': 1764964836.591848, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788739') HTTP/1.1" 204 0
2025-12-05 20:00:36 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: bf858f6a-7a47-53fd-9bc6-83336e169fe9 with identifier: 1788738
2025-12-05 20:00:36 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: bf858f6a-7a47-53fd-9bc6-83336e169fe9 with identifier: 1788738
2025-12-05 20:00:36 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: bf858f6a-7a47-53fd-9bc6-83336e169fe9 with identifier: 1788738 to the API
2025-12-05 20:00:36 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:00:36 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:00:37 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/633/31ccd8adad4c473b89f9d42ab3ed9978/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 96,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'gas',
 'id': 'bf858f6a-7a47-53fd-9bc6-83336e169fe9',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SEAT',
 'model': 'LEON',
 'odometer': 107201,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_bf858f6a-7a47-53fd-9bc6-83336e169fe9/bf858f6a-7a47-53fd-9bc6-83336e169fe9_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 382299,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/633/31ccd8adad4c473b89f9d42ab3ed9978/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-08-06',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SEAT LEON ST 1.5 TGI 96kW DSG-7 S&S Xcellence',
 'transmission': 'manual',
 'trim': 'ST 1.5 TGI 96kW DSG-7 S&S Xcellence',
 'vin': None,
 'year': 2020}
2025-12-05 20:00:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788738%27 HTTP/1.1" 200 None
2025-12-05 20:00:37 [ayvens.es] INFO: Saving data for 1788738: {'created_time': 1764964837.027218, 'last_price_update_time': 1764964837.02723, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788738') HTTP/1.1" 204 0
2025-12-05 20:00:37 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/907/443211875eaa4e5eb6f7ebdff333ef46/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:37 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:37 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:37 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:37 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b289ee10>
2025-12-05 20:00:37 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:37 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:37 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1969
2025-12-05 20:00:37 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:37 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:37 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:37 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_d82f5644-1847-5752-91f1-afeccfa71fb1/d82f5644-1847-5752-91f1-afeccfa71fb1_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F075%252Fb2686d0d08fc4512b0b8224363447e5e%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:37 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/075/b2686d0d08fc4512b0b8224363447e5e/images/large/1.jpg saved to cache
2025-12-05 20:00:37 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item d82f5644-1847-5752-91f1-afeccfa71fb1
2025-12-05 20:00:37 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: d82f5644-1847-5752-91f1-afeccfa71fb1, skipping ID generation
2025-12-05 20:00:37 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item d82f5644-1847-5752-91f1-afeccfa71fb1 with scrape type 1
2025-12-05 20:00:37 [crawlers.middlewares.photo_download_middleware] INFO: Removing item d82f5644-1847-5752-91f1-afeccfa71fb1 from tracking dictionaries
2025-12-05 20:00:37 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item d82f5644-1847-5752-91f1-afeccfa71fb1 sending to next pipeline
2025-12-05 20:00:37 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: d82f5644-1847-5752-91f1-afeccfa71fb1, identifier: 1788736
2025-12-05 20:00:37 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/327/5f42d39bfd2f4da4984101230f581da9/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:37 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/398/23ecebe217c24c009ace313e6cd39319/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:37 [ayvens.es] INFO: Found listing with ID: 1788757
2025-12-05 20:00:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788757%27 HTTP/1.1" 200 None
2025-12-05 20:00:37 [ayvens.es] INFO: Scrape type for 1788757: 1
2025-12-05 20:00:37 [ayvens.es] INFO: Extracting data for car ID: 1788757
2025-12-05 20:00:37 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: d0eb57d9-d3bf-5971-a676-f61bd60b7cae with identifier: 1788757
2025-12-05 20:00:37 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item d0eb57d9-d3bf-5971-a676-f61bd60b7cae with scrape type 1
2025-12-05 20:00:37 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item d0eb57d9-d3bf-5971-a676-f61bd60b7cae
2025-12-05 20:00:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F023%252F1ab4b01e62d34b93b3114cc22b6161a0%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:37 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/023/1ab4b01e62d34b93b3114cc22b6161a0/images/large/1.jpg not downloaded yet
2025-12-05 20:00:37 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: d82f5644-1847-5752-91f1-afeccfa71fb1 with identifier: 1788736
2025-12-05 20:00:37 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:00:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:00:37 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: d82f5644-1847-5752-91f1-afeccfa71fb1 with identifier: 1788736
2025-12-05 20:00:37 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'metallic', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'peugeot', 'model': '3008', 'trim': '1.5l bluehdi 96kw (130cv) s&s gt line **venta estado - revisar peritacion**', 'title': 'peugeot 3008 1.5l bluehdi 96kw (130cv) s&s gt line **venta estado - revisar peritacion**', 'price': 0, 'odometer': 208865, 'odometer_reading_unit': 'km', 'registration_date': '2018-09-27', 'year': 2018, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.5l', 'c_o_2_emission_value': 109, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'd82f5644-1847-5752-91f1-afeccfa71fb1', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_d82f5644-1847-5752-91f1-afeccfa71fb1/d82f5644-1847-5752-91f1-afeccfa71fb1_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/075/b2686d0d08fc4512b0b8224363447e5e/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 469411}], 'current_year_diff': 7, 'age_in_months': 86, 'make_model': 'peugeot 3008', 'sanitized_make': 'peugeot', 'odometer_in_km': 208865}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 86,
     'auction_title': '',
     'c_o_2_emission_value': 109,
     'color': 'metallic',
     'currency': 'eur',
     'current_year_diff': 7,
     'doors_number': 5,
     'engine': '1.5l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': 'd82f5644-1847-5752-91f1-afeccfa71fb1',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'peugeot',
     'make_model': 'peugeot 3008',
     'model': '3008',
     'odometer': 208865,
     'odometer_in_km': 208865,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_d82f5644-1847-5752-91f1-afeccfa71fb1/d82f5644-1847-5752-91f1-afeccfa71fb1_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 469411,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/075/b2686d0d08fc4512b0b8224363447e5e/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2018-09-27',
     'sanitized_make': 'peugeot',
     'seats_number': 5,
     'seller_name': None,
     'title': 'peugeot 3008 1.5l bluehdi 96kw (130cv) s&s gt line **venta '
              'estado - revisar peritacion**',
     'transmission': 'manual',
     'trim': '1.5l bluehdi 96kw (130cv) s&s gt line **venta estado - '
             'revisar peritacion**',
     'vin': None,
     'year': 2018}
2025-12-05 20:00:37 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE failed for item: d82f5644-1847-5752-91f1-afeccfa71fb1 spider: ayvens.es
2025-12-05 20:00:37 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: d82f5644-1847-5752-91f1-afeccfa71fb1 due to rule: high_mileage.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'metallic', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'peugeot', 'model': '3008', 'trim': '1.5l bluehdi 96kw (130cv) s&s gt line **venta estado - revisar peritacion**', 'title': 'peugeot 3008 1.5l bluehdi 96kw (130cv) s&s gt line **venta estado - revisar peritacion**', 'price': 0, 'odometer': 208865, 'odometer_reading_unit': 'km', 'registration_date': '2018-09-27', 'year': 2018, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.5l', 'c_o_2_emission_value': 109, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'd82f5644-1847-5752-91f1-afeccfa71fb1', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_d82f5644-1847-5752-91f1-afeccfa71fb1/d82f5644-1847-5752-91f1-afeccfa71fb1_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/075/b2686d0d08fc4512b0b8224363447e5e/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 469411}], 'current_year_diff': 7, 'age_in_months': 86, 'make_model': 'peugeot 3008', 'sanitized_make': 'peugeot', 'odometer_in_km': 208865}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 86,
     'auction_title': '',
     'c_o_2_emission_value': 109,
     'color': 'metallic',
     'currency': 'eur',
     'current_year_diff': 7,
     'doors_number': 5,
     'engine': '1.5l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': 'd82f5644-1847-5752-91f1-afeccfa71fb1',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'peugeot',
     'make_model': 'peugeot 3008',
     'model': '3008',
     'odometer': 208865,
     'odometer_in_km': 208865,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_d82f5644-1847-5752-91f1-afeccfa71fb1/d82f5644-1847-5752-91f1-afeccfa71fb1_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 469411,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/075/b2686d0d08fc4512b0b8224363447e5e/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2018-09-27',
     'sanitized_make': 'peugeot',
     'seats_number': 5,
     'seller_name': None,
     'title': 'peugeot 3008 1.5l bluehdi 96kw (130cv) s&s gt line **venta '
              'estado - revisar peritacion**',
     'transmission': 'manual',
     'trim': '1.5l bluehdi 96kw (130cv) s&s gt line **venta estado - '
             'revisar peritacion**',
     'vin': None,
     'year': 2018}
{'auction_title': '',
 'c_o_2_emission_value': 109,
 'color': 'Metallic',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'd82f5644-1847-5752-91f1-afeccfa71fb1',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': '3008',
 'odometer': 208865,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_d82f5644-1847-5752-91f1-afeccfa71fb1/d82f5644-1847-5752-91f1-afeccfa71fb1_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 469411,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/075/b2686d0d08fc4512b0b8224363447e5e/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2018-09-27',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT 3008 1.5L BlueHDi 96kW (130CV) S&S GT Line **VENTA ESTADO - '
          'REVISAR PERITACION**',
 'transmission': 'manual',
 'trim': '1.5L BlueHDi 96kW (130CV) S&S GT Line **VENTA ESTADO - REVISAR '
         'PERITACION**',
 'vin': None,
 'year': 2018}
2025-12-05 20:00:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788736%27 HTTP/1.1" 200 None
2025-12-05 20:00:37 [ayvens.es] INFO: Saving data for 1788736: {'created_time': 1764964837.911937, 'last_price_update_time': 1764964837.911947, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788736') HTTP/1.1" 204 0
2025-12-05 20:00:38 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/331/30582bd8c7c64080a21e0c55a2f0181d/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:38 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/893/b6d4c28d896d4f5da7ff1fab4a9b33b2/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:38 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/980/a1224eea2b5f405d893728d9425d0e3f/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:38 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:38 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:38 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:38 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0531c10>
2025-12-05 20:00:38 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:38 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:38 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:00:38 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:38 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:38 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:38 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_323c7d47-482a-5e80-8fad-70199a8a392b/323c7d47-482a-5e80-8fad-70199a8a392b_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F580%252F5ebf9b678fce465f9879db0b6783cf91%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:38 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/580/5ebf9b678fce465f9879db0b6783cf91/images/large/1.jpg saved to cache
2025-12-05 20:00:38 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 323c7d47-482a-5e80-8fad-70199a8a392b
2025-12-05 20:00:38 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 323c7d47-482a-5e80-8fad-70199a8a392b, skipping ID generation
2025-12-05 20:00:38 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 323c7d47-482a-5e80-8fad-70199a8a392b with scrape type 1
2025-12-05 20:00:38 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 323c7d47-482a-5e80-8fad-70199a8a392b from tracking dictionaries
2025-12-05 20:00:38 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 323c7d47-482a-5e80-8fad-70199a8a392b sending to next pipeline
2025-12-05 20:00:38 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 323c7d47-482a-5e80-8fad-70199a8a392b, identifier: 1788737
2025-12-05 20:00:38 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:38 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:38 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:38 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b053f3d0>
2025-12-05 20:00:38 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:38 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:38 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:00:38 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:38 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:38 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:38 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_879c64c5-9a6c-53c0-a91e-a73a4704d491/879c64c5-9a6c-53c0-a91e-a73a4704d491_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F788%252F0024ef09da274917a7c417d43415be1f%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:38 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/788/0024ef09da274917a7c417d43415be1f/images/large/1.jpg saved to cache
2025-12-05 20:00:38 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 879c64c5-9a6c-53c0-a91e-a73a4704d491
2025-12-05 20:00:38 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 879c64c5-9a6c-53c0-a91e-a73a4704d491, skipping ID generation
2025-12-05 20:00:38 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 879c64c5-9a6c-53c0-a91e-a73a4704d491 with scrape type 1
2025-12-05 20:00:38 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 879c64c5-9a6c-53c0-a91e-a73a4704d491 from tracking dictionaries
2025-12-05 20:00:38 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 879c64c5-9a6c-53c0-a91e-a73a4704d491 sending to next pipeline
2025-12-05 20:00:38 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 879c64c5-9a6c-53c0-a91e-a73a4704d491, identifier: 1788740
2025-12-05 20:00:38 [ayvens.es] INFO: Found listing with ID: 1788758
2025-12-05 20:00:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788758%27 HTTP/1.1" 200 None
2025-12-05 20:00:38 [ayvens.es] INFO: Scrape type for 1788758: 1
2025-12-05 20:00:38 [ayvens.es] INFO: Extracting data for car ID: 1788758
2025-12-05 20:00:38 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: ddd41936-fe7a-5eea-b092-b77e5bf095c8 with identifier: 1788758
2025-12-05 20:00:38 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ddd41936-fe7a-5eea-b092-b77e5bf095c8 with scrape type 1
2025-12-05 20:00:38 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item ddd41936-fe7a-5eea-b092-b77e5bf095c8
2025-12-05 20:00:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F488%252F422327244e1c4d5b945c27b0e9cce062%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:38 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/488/422327244e1c4d5b945c27b0e9cce062/images/large/1.jpg not downloaded yet
2025-12-05 20:00:38 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 323c7d47-482a-5e80-8fad-70199a8a392b with identifier: 1788737
2025-12-05 20:00:38 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:00:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:00:39 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:00:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:00:39 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 323c7d47-482a-5e80-8fad-70199a8a392b with identifier: 1788737
2025-12-05 20:00:39 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 323c7d47-482a-5e80-8fad-70199a8a392b with identifier: 1788737 to the API
2025-12-05 20:00:39 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:00:39 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:00:39 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/580/5ebf9b678fce465f9879db0b6783cf91/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 110,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '323c7d47-482a-5e80-8fad-70199a8a392b',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SEAT',
 'model': 'ARONA',
 'odometer': 40837,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_323c7d47-482a-5e80-8fad-70199a8a392b/323c7d47-482a-5e80-8fad-70199a8a392b_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 356870,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/580/5ebf9b678fce465f9879db0b6783cf91/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-07-06',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SEAT ARONA 1.0 TSI 85kW (115CV) Style Edition Eco',
 'transmission': 'manual',
 'trim': '1.0 TSI 85kW (115CV) Style Edition Eco',
 'vin': None,
 'year': 2020}
2025-12-05 20:00:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788737%27 HTTP/1.1" 200 None
2025-12-05 20:00:39 [ayvens.es] INFO: Saving data for 1788737: {'created_time': 1764964839.47125, 'last_price_update_time': 1764964839.471262, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788737') HTTP/1.1" 204 0
2025-12-05 20:00:39 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 879c64c5-9a6c-53c0-a91e-a73a4704d491 with identifier: 1788740
2025-12-05 20:00:39 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 879c64c5-9a6c-53c0-a91e-a73a4704d491 with identifier: 1788740
2025-12-05 20:00:39 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 879c64c5-9a6c-53c0-a91e-a73a4704d491 with identifier: 1788740 to the API
2025-12-05 20:00:39 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:00:39 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:00:39 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/788/0024ef09da274917a7c417d43415be1f/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 96,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'gas',
 'id': '879c64c5-9a6c-53c0-a91e-a73a4704d491',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SEAT',
 'model': 'LEON',
 'odometer': 95694,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_879c64c5-9a6c-53c0-a91e-a73a4704d491/879c64c5-9a6c-53c0-a91e-a73a4704d491_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 363609,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/788/0024ef09da274917a7c417d43415be1f/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-08-04',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SEAT LEON ST 1.5 TGI 96kW DSG-7 S&S Xcellence',
 'transmission': 'manual',
 'trim': 'ST 1.5 TGI 96kW DSG-7 S&S Xcellence',
 'vin': None,
 'year': 2020}
2025-12-05 20:00:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788740%27 HTTP/1.1" 200 None
2025-12-05 20:00:39 [ayvens.es] INFO: Saving data for 1788740: {'created_time': 1764964839.906755, 'last_price_update_time': 1764964839.906768, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788740') HTTP/1.1" 204 0
2025-12-05 20:00:39 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/319/0c141d56ad1a4ba5aa8049bfc73e0f2c/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:40 [ayvens.es] INFO: Found listing with ID: 1788759
2025-12-05 20:00:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788759%27 HTTP/1.1" 200 None
2025-12-05 20:00:40 [ayvens.es] INFO: Scrape type for 1788759: 1
2025-12-05 20:00:40 [ayvens.es] INFO: Extracting data for car ID: 1788759
2025-12-05 20:00:40 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: dd61a0df-df8f-59a4-aa84-0b8946bab97e with identifier: 1788759
2025-12-05 20:00:40 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item dd61a0df-df8f-59a4-aa84-0b8946bab97e with scrape type 1
2025-12-05 20:00:40 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item dd61a0df-df8f-59a4-aa84-0b8946bab97e
2025-12-05 20:00:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F985%252F8498199a8dc54775a7092e8b4b60cb6a%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:40 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/985/8498199a8dc54775a7092e8b4b60cb6a/images/large/1.jpg not downloaded yet
2025-12-05 20:00:40 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:40 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:40 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:40 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a63610>
2025-12-05 20:00:40 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:40 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:40 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:00:40 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:40 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:40 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:40 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_a3c678ce-869c-5147-a58e-bb4699807d0b/a3c678ce-869c-5147-a58e-bb4699807d0b_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F907%252F443211875eaa4e5eb6f7ebdff333ef46%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:40 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/907/443211875eaa4e5eb6f7ebdff333ef46/images/large/1.jpg saved to cache
2025-12-05 20:00:40 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item a3c678ce-869c-5147-a58e-bb4699807d0b
2025-12-05 20:00:40 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: a3c678ce-869c-5147-a58e-bb4699807d0b, skipping ID generation
2025-12-05 20:00:40 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a3c678ce-869c-5147-a58e-bb4699807d0b with scrape type 1
2025-12-05 20:00:40 [crawlers.middlewares.photo_download_middleware] INFO: Removing item a3c678ce-869c-5147-a58e-bb4699807d0b from tracking dictionaries
2025-12-05 20:00:40 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item a3c678ce-869c-5147-a58e-bb4699807d0b sending to next pipeline
2025-12-05 20:00:40 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: a3c678ce-869c-5147-a58e-bb4699807d0b, identifier: 1788743
2025-12-05 20:00:40 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:40 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:40 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:40 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a81c10>
2025-12-05 20:00:40 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:40 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:41 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:00:41 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:41 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:41 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:41 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_b7c086f0-43e5-5cb6-898a-38aef0e17447/b7c086f0-43e5-5cb6-898a-38aef0e17447_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F327%252F5f42d39bfd2f4da4984101230f581da9%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:41 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/327/5f42d39bfd2f4da4984101230f581da9/images/large/1.jpg saved to cache
2025-12-05 20:00:41 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item b7c086f0-43e5-5cb6-898a-38aef0e17447
2025-12-05 20:00:41 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: b7c086f0-43e5-5cb6-898a-38aef0e17447, skipping ID generation
2025-12-05 20:00:41 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b7c086f0-43e5-5cb6-898a-38aef0e17447 with scrape type 1
2025-12-05 20:00:41 [crawlers.middlewares.photo_download_middleware] INFO: Removing item b7c086f0-43e5-5cb6-898a-38aef0e17447 from tracking dictionaries
2025-12-05 20:00:41 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item b7c086f0-43e5-5cb6-898a-38aef0e17447 sending to next pipeline
2025-12-05 20:00:41 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: b7c086f0-43e5-5cb6-898a-38aef0e17447, identifier: 1788741
2025-12-05 20:00:41 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:41 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:41 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:41 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a81e90>
2025-12-05 20:00:41 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:41 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:41 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:00:41 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:41 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:41 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:41 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_b9072299-b299-5048-bc40-359724dfebbc/b9072299-b299-5048-bc40-359724dfebbc_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F398%252F23ecebe217c24c009ace313e6cd39319%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:41 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/398/23ecebe217c24c009ace313e6cd39319/images/large/1.jpg saved to cache
2025-12-05 20:00:41 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item b9072299-b299-5048-bc40-359724dfebbc
2025-12-05 20:00:41 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: b9072299-b299-5048-bc40-359724dfebbc, skipping ID generation
2025-12-05 20:00:41 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b9072299-b299-5048-bc40-359724dfebbc with scrape type 1
2025-12-05 20:00:41 [crawlers.middlewares.photo_download_middleware] INFO: Removing item b9072299-b299-5048-bc40-359724dfebbc from tracking dictionaries
2025-12-05 20:00:41 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item b9072299-b299-5048-bc40-359724dfebbc sending to next pipeline
2025-12-05 20:00:41 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: b9072299-b299-5048-bc40-359724dfebbc, identifier: 1788745
2025-12-05 20:00:41 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:41 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:41 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:41 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a43650>
2025-12-05 20:00:41 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:41 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:42 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1971
2025-12-05 20:00:42 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:42 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:42 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:42 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_43c66767-66a7-5d52-91ef-7193825793ca/43c66767-66a7-5d52-91ef-7193825793ca_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F331%252F30582bd8c7c64080a21e0c55a2f0181d%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:42 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/331/30582bd8c7c64080a21e0c55a2f0181d/images/large/1.jpg saved to cache
2025-12-05 20:00:42 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 43c66767-66a7-5d52-91ef-7193825793ca
2025-12-05 20:00:42 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 43c66767-66a7-5d52-91ef-7193825793ca, skipping ID generation
2025-12-05 20:00:42 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 43c66767-66a7-5d52-91ef-7193825793ca with scrape type 1
2025-12-05 20:00:42 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 43c66767-66a7-5d52-91ef-7193825793ca from tracking dictionaries
2025-12-05 20:00:42 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 43c66767-66a7-5d52-91ef-7193825793ca sending to next pipeline
2025-12-05 20:00:42 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 43c66767-66a7-5d52-91ef-7193825793ca, identifier: 1788744
2025-12-05 20:00:42 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:42 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:42 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:42 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a53b10>
2025-12-05 20:00:42 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:42 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:42 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1971
2025-12-05 20:00:42 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:42 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:42 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:42 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_34b30405-e26c-57ed-8abe-3601873e87d8/34b30405-e26c-57ed-8abe-3601873e87d8_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F893%252Fb6d4c28d896d4f5da7ff1fab4a9b33b2%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:42 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/893/b6d4c28d896d4f5da7ff1fab4a9b33b2/images/large/1.jpg saved to cache
2025-12-05 20:00:42 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 34b30405-e26c-57ed-8abe-3601873e87d8
2025-12-05 20:00:42 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 34b30405-e26c-57ed-8abe-3601873e87d8, skipping ID generation
2025-12-05 20:00:42 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 34b30405-e26c-57ed-8abe-3601873e87d8 with scrape type 1
2025-12-05 20:00:42 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 34b30405-e26c-57ed-8abe-3601873e87d8 from tracking dictionaries
2025-12-05 20:00:42 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 34b30405-e26c-57ed-8abe-3601873e87d8 sending to next pipeline
2025-12-05 20:00:42 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 34b30405-e26c-57ed-8abe-3601873e87d8, identifier: 1788747
2025-12-05 20:00:42 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:42 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:42 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:42 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a24ad0>
2025-12-05 20:00:42 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:42 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:42 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:00:42 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:42 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:42 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:43 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_79911923-59d9-542a-be1f-742ee1ad6b22/79911923-59d9-542a-be1f-742ee1ad6b22_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F980%252Fa1224eea2b5f405d893728d9425d0e3f%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:43 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/980/a1224eea2b5f405d893728d9425d0e3f/images/large/1.jpg saved to cache
2025-12-05 20:00:43 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 79911923-59d9-542a-be1f-742ee1ad6b22
2025-12-05 20:00:43 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 79911923-59d9-542a-be1f-742ee1ad6b22, skipping ID generation
2025-12-05 20:00:43 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 79911923-59d9-542a-be1f-742ee1ad6b22 with scrape type 1
2025-12-05 20:00:43 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 79911923-59d9-542a-be1f-742ee1ad6b22 from tracking dictionaries
2025-12-05 20:00:43 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 79911923-59d9-542a-be1f-742ee1ad6b22 sending to next pipeline
2025-12-05 20:00:43 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 79911923-59d9-542a-be1f-742ee1ad6b22, identifier: 1788742
2025-12-05 20:00:43 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: a3c678ce-869c-5147-a58e-bb4699807d0b with identifier: 1788743
2025-12-05 20:00:43 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: a3c678ce-869c-5147-a58e-bb4699807d0b with identifier: 1788743
2025-12-05 20:00:43 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: a3c678ce-869c-5147-a58e-bb4699807d0b with identifier: 1788743 to the API
2025-12-05 20:00:43 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:00:43 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:00:43 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/907/443211875eaa4e5eb6f7ebdff333ef46/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 96,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'gas',
 'id': 'a3c678ce-869c-5147-a58e-bb4699807d0b',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SEAT',
 'model': 'LEON',
 'odometer': 115161,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_a3c678ce-869c-5147-a58e-bb4699807d0b/a3c678ce-869c-5147-a58e-bb4699807d0b_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 344630,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/907/443211875eaa4e5eb6f7ebdff333ef46/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-08-11',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SEAT LEON 1.5 TGI 96kW DSG-7 S&S Xcellence',
 'transmission': 'manual',
 'trim': '1.5 TGI 96kW DSG-7 S&S Xcellence',
 'vin': None,
 'year': 2020}
2025-12-05 20:00:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788743%27 HTTP/1.1" 200 None
2025-12-05 20:00:43 [ayvens.es] INFO: Saving data for 1788743: {'created_time': 1764964843.517135, 'last_price_update_time': 1764964843.517144, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788743') HTTP/1.1" 204 0
2025-12-05 20:00:43 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: b7c086f0-43e5-5cb6-898a-38aef0e17447 with identifier: 1788741
2025-12-05 20:00:43 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: b7c086f0-43e5-5cb6-898a-38aef0e17447 with identifier: 1788741
2025-12-05 20:00:43 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: b7c086f0-43e5-5cb6-898a-38aef0e17447 with identifier: 1788741 to the API
2025-12-05 20:00:43 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:00:43 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:00:43 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/327/5f42d39bfd2f4da4984101230f581da9/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 98,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'gas',
 'id': 'b7c086f0-43e5-5cb6-898a-38aef0e17447',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SEAT',
 'model': 'LEON',
 'odometer': 84989,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_b7c086f0-43e5-5cb6-898a-38aef0e17447/b7c086f0-43e5-5cb6-898a-38aef0e17447_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 330919,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/327/5f42d39bfd2f4da4984101230f581da9/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-09-09',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SEAT LEON 1.5 TGI 96kW St&Sp Xcellence',
 'transmission': 'manual',
 'trim': '1.5 TGI 96kW St&Sp Xcellence',
 'vin': None,
 'year': 2020}
2025-12-05 20:00:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788741%27 HTTP/1.1" 200 None
2025-12-05 20:00:43 [ayvens.es] INFO: Saving data for 1788741: {'created_time': 1764964843.866976, 'last_price_update_time': 1764964843.866986, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788741') HTTP/1.1" 204 0
2025-12-05 20:00:43 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: b9072299-b299-5048-bc40-359724dfebbc with identifier: 1788745
2025-12-05 20:00:43 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: b9072299-b299-5048-bc40-359724dfebbc with identifier: 1788745
2025-12-05 20:00:43 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: b9072299-b299-5048-bc40-359724dfebbc with identifier: 1788745 to the API
2025-12-05 20:00:43 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:00:44 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:00:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/398/23ecebe217c24c009ace313e6cd39319/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 96,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'gas',
 'id': 'b9072299-b299-5048-bc40-359724dfebbc',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SKODA',
 'model': 'OCTAVIA',
 'odometer': 116581,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_b9072299-b299-5048-bc40-359724dfebbc/b9072299-b299-5048-bc40-359724dfebbc_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 322195,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/398/23ecebe217c24c009ace313e6cd39319/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-07-27',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SKODA OCTAVIA Combi 1.5 TGI 96kW(130CV) Ambit. CNG DSG',
 'transmission': 'manual',
 'trim': 'Combi 1.5 TGI 96kW(130CV) Ambit. CNG DSG',
 'vin': None,
 'year': 2020}
2025-12-05 20:00:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788745%27 HTTP/1.1" 200 None
2025-12-05 20:00:44 [ayvens.es] INFO: Saving data for 1788745: {'created_time': 1764964844.244172, 'last_price_update_time': 1764964844.244182, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788745') HTTP/1.1" 204 0
2025-12-05 20:00:44 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 43c66767-66a7-5d52-91ef-7193825793ca with identifier: 1788744
2025-12-05 20:00:44 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 43c66767-66a7-5d52-91ef-7193825793ca with identifier: 1788744
2025-12-05 20:00:44 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 43c66767-66a7-5d52-91ef-7193825793ca with identifier: 1788744 to the API
2025-12-05 20:00:44 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:00:44 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:00:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/331/30582bd8c7c64080a21e0c55a2f0181d/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 96,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'gas',
 'id': '43c66767-66a7-5d52-91ef-7193825793ca',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SEAT',
 'model': 'LEON',
 'odometer': 88457,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_43c66767-66a7-5d52-91ef-7193825793ca/43c66767-66a7-5d52-91ef-7193825793ca_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 334494,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/331/30582bd8c7c64080a21e0c55a2f0181d/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-08-05',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SEAT LEON ST 1.5 TGI 96kW DSG-7 S&S Xcellence',
 'transmission': 'manual',
 'trim': 'ST 1.5 TGI 96kW DSG-7 S&S Xcellence',
 'vin': None,
 'year': 2020}
2025-12-05 20:00:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788744%27 HTTP/1.1" 200 None
2025-12-05 20:00:44 [ayvens.es] INFO: Saving data for 1788744: {'created_time': 1764964844.631862, 'last_price_update_time': 1764964844.631872, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788744') HTTP/1.1" 204 0
2025-12-05 20:00:44 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 34b30405-e26c-57ed-8abe-3601873e87d8 with identifier: 1788747
2025-12-05 20:00:44 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 34b30405-e26c-57ed-8abe-3601873e87d8 with identifier: 1788747
2025-12-05 20:00:44 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 34b30405-e26c-57ed-8abe-3601873e87d8 with identifier: 1788747 to the API
2025-12-05 20:00:44 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:00:44 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:00:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/893/b6d4c28d896d4f5da7ff1fab4a9b33b2/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 98,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'gas',
 'id': '34b30405-e26c-57ed-8abe-3601873e87d8',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SEAT',
 'model': 'LEON',
 'odometer': 73348,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_34b30405-e26c-57ed-8abe-3601873e87d8/34b30405-e26c-57ed-8abe-3601873e87d8_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 348314,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/893/b6d4c28d896d4f5da7ff1fab4a9b33b2/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-09-21',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SEAT LEON 1.5 TGI 96kW St&Sp Xcellence',
 'transmission': 'manual',
 'trim': '1.5 TGI 96kW St&Sp Xcellence',
 'vin': None,
 'year': 2020}
2025-12-05 20:00:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788747%27 HTTP/1.1" 200 None
2025-12-05 20:00:45 [ayvens.es] INFO: Saving data for 1788747: {'created_time': 1764964845.010142, 'last_price_update_time': 1764964845.010155, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788747') HTTP/1.1" 204 0
2025-12-05 20:00:45 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 79911923-59d9-542a-be1f-742ee1ad6b22 with identifier: 1788742
2025-12-05 20:00:45 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 79911923-59d9-542a-be1f-742ee1ad6b22 with identifier: 1788742
2025-12-05 20:00:45 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 79911923-59d9-542a-be1f-742ee1ad6b22 with identifier: 1788742 to the API
2025-12-05 20:00:45 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:00:46 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:00:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/980/a1224eea2b5f405d893728d9425d0e3f/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 96,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'gas',
 'id': '79911923-59d9-542a-be1f-742ee1ad6b22',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SEAT',
 'model': 'LEON',
 'odometer': 141438,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_79911923-59d9-542a-be1f-742ee1ad6b22/79911923-59d9-542a-be1f-742ee1ad6b22_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 329002,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/980/a1224eea2b5f405d893728d9425d0e3f/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-07-28',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SEAT LEON ST 1.5 TGI 96kW DSG-7 S&S Xcellence',
 'transmission': 'manual',
 'trim': 'ST 1.5 TGI 96kW DSG-7 S&S Xcellence',
 'vin': None,
 'year': 2020}
2025-12-05 20:00:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788742%27 HTTP/1.1" 200 None
2025-12-05 20:00:47 [ayvens.es] INFO: Saving data for 1788742: {'created_time': 1764964847.067796, 'last_price_update_time': 1764964847.067806, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788742') HTTP/1.1" 204 0
2025-12-05 20:00:47 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/944/47b913861add4db28da3355976d19c19/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:47 [ayvens.es] INFO: Found listing with ID: 1788760
2025-12-05 20:00:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788760%27 HTTP/1.1" 200 None
2025-12-05 20:00:47 [ayvens.es] INFO: Scrape type for 1788760: 1
2025-12-05 20:00:47 [ayvens.es] INFO: Extracting data for car ID: 1788760
2025-12-05 20:00:47 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 6b760c32-fa26-597d-80d0-9052a5aba827 with identifier: 1788760
2025-12-05 20:00:47 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6b760c32-fa26-597d-80d0-9052a5aba827 with scrape type 1
2025-12-05 20:00:47 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 6b760c32-fa26-597d-80d0-9052a5aba827
2025-12-05 20:00:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F948%252F4a249ff5e3cc4671b2f0b04d4aa075ff%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:47 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/948/4a249ff5e3cc4671b2f0b04d4aa075ff/images/large/1.jpg not downloaded yet
2025-12-05 20:00:47 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/866/adae37cfb08344809baf7cf88779f315/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:47 [ayvens.es] INFO: Found listing with ID: 1788761
2025-12-05 20:00:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788761%27 HTTP/1.1" 200 None
2025-12-05 20:00:47 [ayvens.es] INFO: Scrape type for 1788761: 1
2025-12-05 20:00:47 [ayvens.es] INFO: Extracting data for car ID: 1788761
2025-12-05 20:00:47 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 1067cb2b-6756-531f-b430-f5c2bf849c5a with identifier: 1788761
2025-12-05 20:00:47 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 1067cb2b-6756-531f-b430-f5c2bf849c5a with scrape type 1
2025-12-05 20:00:47 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 1067cb2b-6756-531f-b430-f5c2bf849c5a
2025-12-05 20:00:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F650%252F45d30630f4144a909d28b0a9c1da36d9%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:47 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/650/45d30630f4144a909d28b0a9c1da36d9/images/large/1.jpg not downloaded yet
2025-12-05 20:00:47 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:47 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:47 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:47 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b04ce010>
2025-12-05 20:00:47 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:47 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:47 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:00:47 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:47 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:47 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:47 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_ba23f778-25cd-5eb2-9d2d-47c45a866f9f/ba23f778-25cd-5eb2-9d2d-47c45a866f9f_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F319%252F0c141d56ad1a4ba5aa8049bfc73e0f2c%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:47 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/319/0c141d56ad1a4ba5aa8049bfc73e0f2c/images/large/1.jpg saved to cache
2025-12-05 20:00:47 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item ba23f778-25cd-5eb2-9d2d-47c45a866f9f
2025-12-05 20:00:47 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: ba23f778-25cd-5eb2-9d2d-47c45a866f9f, skipping ID generation
2025-12-05 20:00:47 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ba23f778-25cd-5eb2-9d2d-47c45a866f9f with scrape type 1
2025-12-05 20:00:47 [crawlers.middlewares.photo_download_middleware] INFO: Removing item ba23f778-25cd-5eb2-9d2d-47c45a866f9f from tracking dictionaries
2025-12-05 20:00:47 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item ba23f778-25cd-5eb2-9d2d-47c45a866f9f sending to next pipeline
2025-12-05 20:00:47 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: ba23f778-25cd-5eb2-9d2d-47c45a866f9f, identifier: 1788748
2025-12-05 20:00:47 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/591/55e129043e144cb3abf69e0b9d14c367/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:47 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/575/4e6c70c31cb044f584e682d6a6c7d864/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:47 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/119/5726d2f38e1c491293a7f63dc64d5950/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:47 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/263/6fbfe82b8c2b41058b1e7ac6d094e687/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:47 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: ba23f778-25cd-5eb2-9d2d-47c45a866f9f with identifier: 1788748
2025-12-05 20:00:47 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: ba23f778-25cd-5eb2-9d2d-47c45a866f9f with identifier: 1788748
2025-12-05 20:00:47 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: ba23f778-25cd-5eb2-9d2d-47c45a866f9f with identifier: 1788748 to the API
2025-12-05 20:00:47 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:00:47 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:00:48 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/319/0c141d56ad1a4ba5aa8049bfc73e0f2c/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 96,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'gas',
 'id': 'ba23f778-25cd-5eb2-9d2d-47c45a866f9f',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SKODA',
 'model': 'OCTAVIA',
 'odometer': 139141,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ba23f778-25cd-5eb2-9d2d-47c45a866f9f/ba23f778-25cd-5eb2-9d2d-47c45a866f9f_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 374123,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/319/0c141d56ad1a4ba5aa8049bfc73e0f2c/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-10-19',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SKODA OCTAVIA Combi 1.5 TGI 96kW(130CV) Ambit. CNG DSG',
 'transmission': 'manual',
 'trim': 'Combi 1.5 TGI 96kW(130CV) Ambit. CNG DSG',
 'vin': None,
 'year': 2020}
2025-12-05 20:00:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788748%27 HTTP/1.1" 200 None
2025-12-05 20:00:48 [ayvens.es] INFO: Saving data for 1788748: {'created_time': 1764964848.115167, 'last_price_update_time': 1764964848.115176, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788748') HTTP/1.1" 204 0
2025-12-05 20:00:48 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/575/addcb1c702614f949522fc5c38893a15/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:48 [ayvens.es] INFO: Found listing with ID: 1788762
2025-12-05 20:00:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788762%27 HTTP/1.1" 200 None
2025-12-05 20:00:48 [ayvens.es] INFO: Scrape type for 1788762: 1
2025-12-05 20:00:48 [ayvens.es] INFO: Extracting data for car ID: 1788762
2025-12-05 20:00:48 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c with identifier: 1788762
2025-12-05 20:00:48 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c with scrape type 1
2025-12-05 20:00:48 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c
2025-12-05 20:00:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F179%252F2d75fee09b0d4894aee9c28f24e11f1b%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:48 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/179/2d75fee09b0d4894aee9c28f24e11f1b/images/large/1.jpg not downloaded yet
2025-12-05 20:00:48 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:48 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:48 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:48 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b06c7990>
2025-12-05 20:00:48 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:48 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:48 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1969
2025-12-05 20:00:48 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:48 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:48 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:48 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_116c9546-b2b2-5b81-a633-fe0f0e82c12a/116c9546-b2b2-5b81-a633-fe0f0e82c12a_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F944%252F47b913861add4db28da3355976d19c19%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:48 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/944/47b913861add4db28da3355976d19c19/images/large/1.jpg saved to cache
2025-12-05 20:00:48 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 116c9546-b2b2-5b81-a633-fe0f0e82c12a
2025-12-05 20:00:48 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 116c9546-b2b2-5b81-a633-fe0f0e82c12a, skipping ID generation
2025-12-05 20:00:48 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 116c9546-b2b2-5b81-a633-fe0f0e82c12a with scrape type 1
2025-12-05 20:00:48 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 116c9546-b2b2-5b81-a633-fe0f0e82c12a from tracking dictionaries
2025-12-05 20:00:48 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 116c9546-b2b2-5b81-a633-fe0f0e82c12a sending to next pipeline
2025-12-05 20:00:48 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 116c9546-b2b2-5b81-a633-fe0f0e82c12a, identifier: 1788749
2025-12-05 20:00:48 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:48 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:48 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:48 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b06e2a50>
2025-12-05 20:00:48 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:48 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:49 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:00:49 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:49 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:49 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:49 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_6e43078d-5ac1-5dd1-b035-99747b2d3142/6e43078d-5ac1-5dd1-b035-99747b2d3142_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F866%252Fadae37cfb08344809baf7cf88779f315%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:49 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/866/adae37cfb08344809baf7cf88779f315/images/large/1.jpg saved to cache
2025-12-05 20:00:49 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 6e43078d-5ac1-5dd1-b035-99747b2d3142
2025-12-05 20:00:49 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 6e43078d-5ac1-5dd1-b035-99747b2d3142, skipping ID generation
2025-12-05 20:00:49 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6e43078d-5ac1-5dd1-b035-99747b2d3142 with scrape type 1
2025-12-05 20:00:49 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 6e43078d-5ac1-5dd1-b035-99747b2d3142 from tracking dictionaries
2025-12-05 20:00:49 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 6e43078d-5ac1-5dd1-b035-99747b2d3142 sending to next pipeline
2025-12-05 20:00:49 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 6e43078d-5ac1-5dd1-b035-99747b2d3142, identifier: 1788752
2025-12-05 20:00:49 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/809/18e3a6f9aaf64244b3cae0ff5e320441/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:49 [ayvens.es] INFO: Found listing with ID: 1788763
2025-12-05 20:00:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788763%27 HTTP/1.1" 200 None
2025-12-05 20:00:49 [ayvens.es] INFO: Scrape type for 1788763: 1
2025-12-05 20:00:49 [ayvens.es] INFO: Extracting data for car ID: 1788763
2025-12-05 20:00:49 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4 with identifier: 1788763
2025-12-05 20:00:49 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4 with scrape type 1
2025-12-05 20:00:49 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4
2025-12-05 20:00:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F974%252F7ce53bcd81cd4ef3bb838dd0adbb8c60%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:49 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/974/7ce53bcd81cd4ef3bb838dd0adbb8c60/images/large/1.jpg not downloaded yet
2025-12-05 20:00:49 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 116c9546-b2b2-5b81-a633-fe0f0e82c12a with identifier: 1788749
2025-12-05 20:00:49 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:00:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:00:49 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 116c9546-b2b2-5b81-a633-fe0f0e82c12a with identifier: 1788749
2025-12-05 20:00:49 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 116c9546-b2b2-5b81-a633-fe0f0e82c12a with identifier: 1788749 to the API
2025-12-05 20:00:49 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:00:49 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:00:50 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/944/47b913861add4db28da3355976d19c19/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 86,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.8L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'hybrid',
 'id': '116c9546-b2b2-5b81-a633-fe0f0e82c12a',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'TOYOTA',
 'model': 'C-HR',
 'odometer': 58375,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_116c9546-b2b2-5b81-a633-fe0f0e82c12a/116c9546-b2b2-5b81-a633-fe0f0e82c12a_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 497597,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/944/47b913861add4db28da3355976d19c19/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-06-04',
 'seats_number': 5,
 'seller_name': None,
 'title': 'TOYOTA C-HR 1.8 125H Advance',
 'transmission': 'manual',
 'trim': '1.8 125H Advance',
 'vin': None,
 'year': 2019}
2025-12-05 20:00:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788749%27 HTTP/1.1" 200 None
2025-12-05 20:00:50 [ayvens.es] INFO: Saving data for 1788749: {'created_time': 1764964850.03543, 'last_price_update_time': 1764964850.035441, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788749') HTTP/1.1" 204 0
2025-12-05 20:00:50 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 6e43078d-5ac1-5dd1-b035-99747b2d3142 with identifier: 1788752
2025-12-05 20:00:50 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 6e43078d-5ac1-5dd1-b035-99747b2d3142 with identifier: 1788752
2025-12-05 20:00:50 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 6e43078d-5ac1-5dd1-b035-99747b2d3142 with identifier: 1788752 to the API
2025-12-05 20:00:50 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:00:50 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:00:50 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/866/adae37cfb08344809baf7cf88779f315/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 113,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '6e43078d-5ac1-5dd1-b035-99747b2d3142',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SEAT',
 'model': 'LEON',
 'odometer': 124983,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_6e43078d-5ac1-5dd1-b035-99747b2d3142/6e43078d-5ac1-5dd1-b035-99747b2d3142_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 314613,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/866/adae37cfb08344809baf7cf88779f315/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-10-15',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SEAT LEON 2.0 TDI 85kW S&S Style Go',
 'transmission': 'manual',
 'trim': '2.0 TDI 85kW S&S Style Go',
 'vin': None,
 'year': 2021}
2025-12-05 20:00:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788752%27 HTTP/1.1" 200 None
2025-12-05 20:00:50 [ayvens.es] INFO: Saving data for 1788752: {'created_time': 1764964850.457741, 'last_price_update_time': 1764964850.457752, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788752') HTTP/1.1" 204 0
2025-12-05 20:00:50 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:50 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:50 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:50 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b8c150>
2025-12-05 20:00:50 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:50 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:50 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:00:50 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:50 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:50 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:50 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_b7bda7b1-f7a2-568a-b818-31b0ac309fc1/b7bda7b1-f7a2-568a-b818-31b0ac309fc1_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F591%252F55e129043e144cb3abf69e0b9d14c367%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:50 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/591/55e129043e144cb3abf69e0b9d14c367/images/large/1.jpg saved to cache
2025-12-05 20:00:50 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item b7bda7b1-f7a2-568a-b818-31b0ac309fc1
2025-12-05 20:00:50 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: b7bda7b1-f7a2-568a-b818-31b0ac309fc1, skipping ID generation
2025-12-05 20:00:50 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b7bda7b1-f7a2-568a-b818-31b0ac309fc1 with scrape type 1
2025-12-05 20:00:50 [crawlers.middlewares.photo_download_middleware] INFO: Removing item b7bda7b1-f7a2-568a-b818-31b0ac309fc1 from tracking dictionaries
2025-12-05 20:00:50 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item b7bda7b1-f7a2-568a-b818-31b0ac309fc1 sending to next pipeline
2025-12-05 20:00:51 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: b7bda7b1-f7a2-568a-b818-31b0ac309fc1, identifier: 1788755
2025-12-05 20:00:51 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:51 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:51 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:51 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0c4cfd0>
2025-12-05 20:00:51 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:51 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:51 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:00:51 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:51 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:51 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:51 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_e957512c-cc59-50fd-b9c0-8801484aa268/e957512c-cc59-50fd-b9c0-8801484aa268_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F575%252F4e6c70c31cb044f584e682d6a6c7d864%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:51 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/575/4e6c70c31cb044f584e682d6a6c7d864/images/large/1.jpg saved to cache
2025-12-05 20:00:51 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item e957512c-cc59-50fd-b9c0-8801484aa268
2025-12-05 20:00:51 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: e957512c-cc59-50fd-b9c0-8801484aa268, skipping ID generation
2025-12-05 20:00:51 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e957512c-cc59-50fd-b9c0-8801484aa268 with scrape type 1
2025-12-05 20:00:51 [crawlers.middlewares.photo_download_middleware] INFO: Removing item e957512c-cc59-50fd-b9c0-8801484aa268 from tracking dictionaries
2025-12-05 20:00:51 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item e957512c-cc59-50fd-b9c0-8801484aa268 sending to next pipeline
2025-12-05 20:00:51 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: e957512c-cc59-50fd-b9c0-8801484aa268, identifier: 1788751
2025-12-05 20:00:51 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:51 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:51 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:51 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0c5b010>
2025-12-05 20:00:51 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:51 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:51 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:00:51 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:51 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:51 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:51 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_f1c1361e-8b28-54cc-872c-4a7549e871af/f1c1361e-8b28-54cc-872c-4a7549e871af_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F119%252F5726d2f38e1c491293a7f63dc64d5950%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:51 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/119/5726d2f38e1c491293a7f63dc64d5950/images/large/1.jpg saved to cache
2025-12-05 20:00:51 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item f1c1361e-8b28-54cc-872c-4a7549e871af
2025-12-05 20:00:51 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: f1c1361e-8b28-54cc-872c-4a7549e871af, skipping ID generation
2025-12-05 20:00:51 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f1c1361e-8b28-54cc-872c-4a7549e871af with scrape type 1
2025-12-05 20:00:51 [crawlers.middlewares.photo_download_middleware] INFO: Removing item f1c1361e-8b28-54cc-872c-4a7549e871af from tracking dictionaries
2025-12-05 20:00:51 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item f1c1361e-8b28-54cc-872c-4a7549e871af sending to next pipeline
2025-12-05 20:00:52 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: f1c1361e-8b28-54cc-872c-4a7549e871af, identifier: 1788754
2025-12-05 20:00:52 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:52 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:52 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:52 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0df25d0>
2025-12-05 20:00:52 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:52 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:52 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:00:52 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:52 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:52 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:52 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_90be6337-5eee-580b-9438-6b5ee5cce03d/90be6337-5eee-580b-9438-6b5ee5cce03d_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F263%252F6fbfe82b8c2b41058b1e7ac6d094e687%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:52 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/263/6fbfe82b8c2b41058b1e7ac6d094e687/images/large/1.jpg saved to cache
2025-12-05 20:00:52 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 90be6337-5eee-580b-9438-6b5ee5cce03d
2025-12-05 20:00:52 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 90be6337-5eee-580b-9438-6b5ee5cce03d, skipping ID generation
2025-12-05 20:00:52 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 90be6337-5eee-580b-9438-6b5ee5cce03d with scrape type 1
2025-12-05 20:00:52 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 90be6337-5eee-580b-9438-6b5ee5cce03d from tracking dictionaries
2025-12-05 20:00:52 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 90be6337-5eee-580b-9438-6b5ee5cce03d sending to next pipeline
2025-12-05 20:00:52 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 90be6337-5eee-580b-9438-6b5ee5cce03d, identifier: 1788753
2025-12-05 20:00:52 [ayvens.es] INFO: Found listing with ID: 1788764
2025-12-05 20:00:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788764%27 HTTP/1.1" 200 None
2025-12-05 20:00:52 [ayvens.es] INFO: Scrape type for 1788764: 1
2025-12-05 20:00:52 [ayvens.es] INFO: Extracting data for car ID: 1788764
2025-12-05 20:00:52 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 88a958bc-2914-56f4-b2a3-b4a183a78bda with identifier: 1788764
2025-12-05 20:00:52 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 88a958bc-2914-56f4-b2a3-b4a183a78bda with scrape type 1
2025-12-05 20:00:52 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 88a958bc-2914-56f4-b2a3-b4a183a78bda
2025-12-05 20:00:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F678%252F7631af3845104ceca1c9aff6509a5c01%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:52 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/678/7631af3845104ceca1c9aff6509a5c01/images/large/1.jpg not downloaded yet
2025-12-05 20:00:52 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: b7bda7b1-f7a2-568a-b818-31b0ac309fc1 with identifier: 1788755
2025-12-05 20:00:52 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:00:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:00:52 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: b7bda7b1-f7a2-568a-b818-31b0ac309fc1 with identifier: 1788755
2025-12-05 20:00:52 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: b7bda7b1-f7a2-568a-b818-31b0ac309fc1 with identifier: 1788755 to the API
2025-12-05 20:00:52 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:00:52 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:00:52 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/591/55e129043e144cb3abf69e0b9d14c367/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 120,
 'color': 'Azul',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'b7bda7b1-f7a2-568a-b818-31b0ac309fc1',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': '508',
 'odometer': 98691,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_b7bda7b1-f7a2-568a-b818-31b0ac309fc1/b7bda7b1-f7a2-568a-b818-31b0ac309fc1_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 368408,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/591/55e129043e144cb3abf69e0b9d14c367/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-05-17',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT 508 5P Active Pack BlueHDi 130 S&S EAT8',
 'transmission': 'manual',
 'trim': '5P Active Pack BlueHDi 130 S&S EAT8',
 'vin': None,
 'year': 2021}
2025-12-05 20:00:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788755%27 HTTP/1.1" 200 None
2025-12-05 20:00:52 [ayvens.es] INFO: Saving data for 1788755: {'created_time': 1764964852.89509, 'last_price_update_time': 1764964852.895101, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788755') HTTP/1.1" 204 0
2025-12-05 20:00:52 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: e957512c-cc59-50fd-b9c0-8801484aa268 with identifier: 1788751
2025-12-05 20:00:52 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: e957512c-cc59-50fd-b9c0-8801484aa268 with identifier: 1788751
2025-12-05 20:00:53 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: e957512c-cc59-50fd-b9c0-8801484aa268 with identifier: 1788751 to the API
2025-12-05 20:00:53 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:00:53 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:00:53 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/575/4e6c70c31cb044f584e682d6a6c7d864/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 113,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'e957512c-cc59-50fd-b9c0-8801484aa268',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SEAT',
 'model': 'LEON',
 'odometer': 107064,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_e957512c-cc59-50fd-b9c0-8801484aa268/e957512c-cc59-50fd-b9c0-8801484aa268_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 330457,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/575/4e6c70c31cb044f584e682d6a6c7d864/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-04-16',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SEAT LEON 2.0 TDI 85kW S&S Style Go',
 'transmission': 'manual',
 'trim': '2.0 TDI 85kW S&S Style Go',
 'vin': None,
 'year': 2021}
2025-12-05 20:00:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788751%27 HTTP/1.1" 200 None
2025-12-05 20:00:53 [ayvens.es] INFO: Saving data for 1788751: {'created_time': 1764964853.274887, 'last_price_update_time': 1764964853.274899, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788751') HTTP/1.1" 204 0
2025-12-05 20:00:53 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: f1c1361e-8b28-54cc-872c-4a7549e871af with identifier: 1788754
2025-12-05 20:00:53 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:00:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:00:53 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: f1c1361e-8b28-54cc-872c-4a7549e871af with identifier: 1788754
2025-12-05 20:00:53 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: f1c1361e-8b28-54cc-872c-4a7549e871af with identifier: 1788754 to the API
2025-12-05 20:00:53 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:00:53 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:00:53 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/119/5726d2f38e1c491293a7f63dc64d5950/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 31,
 'color': 'Negro',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.6L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'hybrid',
 'id': 'f1c1361e-8b28-54cc-872c-4a7549e871af',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'DS AUTOMOBILES',
 'model': 'DS 7 CROSSBACK',
 'odometer': 30782,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_f1c1361e-8b28-54cc-872c-4a7549e871af/f1c1361e-8b28-54cc-872c-4a7549e871af_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 378553,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/119/5726d2f38e1c491293a7f63dc64d5950/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-02-23',
 'seats_number': 5,
 'seller_name': None,
 'title': 'DS AUTOMOBILES DS 7 CROSSBACK 1.6 E-Tense 225 PERFORMANCE LINE + '
          'Auto',
 'transmission': 'manual',
 'trim': '1.6 E-Tense 225 PERFORMANCE LINE + Auto',
 'vin': None,
 'year': 2021}
2025-12-05 20:00:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788754%27 HTTP/1.1" 200 None
2025-12-05 20:00:53 [ayvens.es] INFO: Saving data for 1788754: {'created_time': 1764964853.644307, 'last_price_update_time': 1764964853.644318, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788754') HTTP/1.1" 204 0
2025-12-05 20:00:53 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 90be6337-5eee-580b-9438-6b5ee5cce03d with identifier: 1788753
2025-12-05 20:00:53 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 90be6337-5eee-580b-9438-6b5ee5cce03d with identifier: 1788753
2025-12-05 20:00:53 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'negro', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'citroën', 'model': 'c3 aircross', 'trim': 'puretech 81kw (110cv) s&s feel pack', 'title': 'citroën c3 aircross puretech 81kw (110cv) s&s feel pack', 'price': 0, 'odometer': 24050, 'odometer_reading_unit': 'km', 'registration_date': '2021-10-08', 'year': 2021, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.2l', 'c_o_2_emission_value': 141, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '90be6337-5eee-580b-9438-6b5ee5cce03d', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_90be6337-5eee-580b-9438-6b5ee5cce03d/90be6337-5eee-580b-9438-6b5ee5cce03d_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/263/6fbfe82b8c2b41058b1e7ac6d094e687/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 365597}], 'current_year_diff': 4, 'age_in_months': 49, 'make_model': 'citroën c3 aircross', 'sanitized_make': 'citron', 'odometer_in_km': 24050}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 49,
     'auction_title': '',
     'c_o_2_emission_value': 141,
     'color': 'negro',
     'currency': 'eur',
     'current_year_diff': 4,
     'doors_number': 5,
     'engine': '1.2l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': '90be6337-5eee-580b-9438-6b5ee5cce03d',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'citroën',
     'make_model': 'citroën c3 aircross',
     'model': 'c3 aircross',
     'odometer': 24050,
     'odometer_in_km': 24050,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_90be6337-5eee-580b-9438-6b5ee5cce03d/90be6337-5eee-580b-9438-6b5ee5cce03d_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 365597,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/263/6fbfe82b8c2b41058b1e7ac6d094e687/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2021-10-08',
     'sanitized_make': 'citron',
     'seats_number': 5,
     'seller_name': None,
     'title': 'citroën c3 aircross puretech 81kw (110cv) s&s feel pack',
     'transmission': 'manual',
     'trim': 'puretech 81kw (110cv) s&s feel pack',
     'vin': None,
     'year': 2021}
2025-12-05 20:00:53 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: PURETECH_FOR_COUNTRY failed for item: 90be6337-5eee-580b-9438-6b5ee5cce03d spider: ayvens.es
2025-12-05 20:00:53 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 90be6337-5eee-580b-9438-6b5ee5cce03d due to rule: puretech_for_country.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'negro', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'citroën', 'model': 'c3 aircross', 'trim': 'puretech 81kw (110cv) s&s feel pack', 'title': 'citroën c3 aircross puretech 81kw (110cv) s&s feel pack', 'price': 0, 'odometer': 24050, 'odometer_reading_unit': 'km', 'registration_date': '2021-10-08', 'year': 2021, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.2l', 'c_o_2_emission_value': 141, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '90be6337-5eee-580b-9438-6b5ee5cce03d', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_90be6337-5eee-580b-9438-6b5ee5cce03d/90be6337-5eee-580b-9438-6b5ee5cce03d_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/263/6fbfe82b8c2b41058b1e7ac6d094e687/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 365597}], 'current_year_diff': 4, 'age_in_months': 49, 'make_model': 'citroën c3 aircross', 'sanitized_make': 'citron', 'odometer_in_km': 24050}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 49,
     'auction_title': '',
     'c_o_2_emission_value': 141,
     'color': 'negro',
     'currency': 'eur',
     'current_year_diff': 4,
     'doors_number': 5,
     'engine': '1.2l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': '90be6337-5eee-580b-9438-6b5ee5cce03d',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'citroën',
     'make_model': 'citroën c3 aircross',
     'model': 'c3 aircross',
     'odometer': 24050,
     'odometer_in_km': 24050,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_90be6337-5eee-580b-9438-6b5ee5cce03d/90be6337-5eee-580b-9438-6b5ee5cce03d_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 365597,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/263/6fbfe82b8c2b41058b1e7ac6d094e687/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2021-10-08',
     'sanitized_make': 'citron',
     'seats_number': 5,
     'seller_name': None,
     'title': 'citroën c3 aircross puretech 81kw (110cv) s&s feel pack',
     'transmission': 'manual',
     'trim': 'puretech 81kw (110cv) s&s feel pack',
     'vin': None,
     'year': 2021}
{'auction_title': '',
 'c_o_2_emission_value': 141,
 'color': 'Negro',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.2L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '90be6337-5eee-580b-9438-6b5ee5cce03d',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'CITROËN',
 'model': 'C3 AIRCROSS',
 'odometer': 24050,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_90be6337-5eee-580b-9438-6b5ee5cce03d/90be6337-5eee-580b-9438-6b5ee5cce03d_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 365597,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/263/6fbfe82b8c2b41058b1e7ac6d094e687/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-10-08',
 'seats_number': 5,
 'seller_name': None,
 'title': 'CITROËN C3 AIRCROSS PureTech 81kW (110CV) S&S Feel Pack',
 'transmission': 'manual',
 'trim': 'PureTech 81kW (110CV) S&S Feel Pack',
 'vin': None,
 'year': 2021}
2025-12-05 20:00:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788753%27 HTTP/1.1" 200 None
2025-12-05 20:00:53 [ayvens.es] INFO: Saving data for 1788753: {'created_time': 1764964853.946706, 'last_price_update_time': 1764964853.94672, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788753') HTTP/1.1" 204 0
2025-12-05 20:00:53 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/023/1ab4b01e62d34b93b3114cc22b6161a0/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:53 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/488/422327244e1c4d5b945c27b0e9cce062/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:53 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:53 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:53 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:53 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0c77c10>
2025-12-05 20:00:53 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:53 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:54 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:00:54 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:54 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:54 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:54 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_9846a71a-949c-5f22-98a6-75322fa13620/9846a71a-949c-5f22-98a6-75322fa13620_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F575%252Faddcb1c702614f949522fc5c38893a15%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:54 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/575/addcb1c702614f949522fc5c38893a15/images/large/1.jpg saved to cache
2025-12-05 20:00:54 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 9846a71a-949c-5f22-98a6-75322fa13620
2025-12-05 20:00:54 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 9846a71a-949c-5f22-98a6-75322fa13620, skipping ID generation
2025-12-05 20:00:54 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9846a71a-949c-5f22-98a6-75322fa13620 with scrape type 1
2025-12-05 20:00:54 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 9846a71a-949c-5f22-98a6-75322fa13620 from tracking dictionaries
2025-12-05 20:00:54 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 9846a71a-949c-5f22-98a6-75322fa13620 sending to next pipeline
2025-12-05 20:00:54 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 9846a71a-949c-5f22-98a6-75322fa13620, identifier: 1788750
2025-12-05 20:00:54 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:54 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:54 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:54 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b05e3a50>
2025-12-05 20:00:54 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:54 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:54 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:00:54 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:54 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:54 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:54 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_786d7a25-8513-559f-bf2d-612205bd215c/786d7a25-8513-559f-bf2d-612205bd215c_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F809%252F18e3a6f9aaf64244b3cae0ff5e320441%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:54 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/809/18e3a6f9aaf64244b3cae0ff5e320441/images/large/1.jpg saved to cache
2025-12-05 20:00:54 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 786d7a25-8513-559f-bf2d-612205bd215c
2025-12-05 20:00:54 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 786d7a25-8513-559f-bf2d-612205bd215c, skipping ID generation
2025-12-05 20:00:54 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 786d7a25-8513-559f-bf2d-612205bd215c with scrape type 1
2025-12-05 20:00:54 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 786d7a25-8513-559f-bf2d-612205bd215c from tracking dictionaries
2025-12-05 20:00:54 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 786d7a25-8513-559f-bf2d-612205bd215c sending to next pipeline
2025-12-05 20:00:54 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 786d7a25-8513-559f-bf2d-612205bd215c, identifier: 1788756
2025-12-05 20:00:54 [ayvens.es] INFO: Found listing with ID: 1788765
2025-12-05 20:00:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788765%27 HTTP/1.1" 200 None
2025-12-05 20:00:54 [ayvens.es] INFO: Scrape type for 1788765: 1
2025-12-05 20:00:54 [ayvens.es] INFO: Extracting data for car ID: 1788765
2025-12-05 20:00:54 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 52f27cf9-37d2-5078-9c54-800687b9a9a5 with identifier: 1788765
2025-12-05 20:00:54 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 52f27cf9-37d2-5078-9c54-800687b9a9a5 with scrape type 1
2025-12-05 20:00:54 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 52f27cf9-37d2-5078-9c54-800687b9a9a5
2025-12-05 20:00:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F774%252Fa4ce057bc3df4fd5a4e74db182dac929%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:54 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/774/a4ce057bc3df4fd5a4e74db182dac929/images/large/1.jpg not downloaded yet
2025-12-05 20:00:54 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 9846a71a-949c-5f22-98a6-75322fa13620 with identifier: 1788750
2025-12-05 20:00:54 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:00:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:00:54 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 9846a71a-949c-5f22-98a6-75322fa13620 with identifier: 1788750
2025-12-05 20:00:55 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 9846a71a-949c-5f22-98a6-75322fa13620 with identifier: 1788750 to the API
2025-12-05 20:00:55 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:00:55 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:00:55 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/575/addcb1c702614f949522fc5c38893a15/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 46,
 'color': 'Rojo',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.3L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'hybrid',
 'id': '9846a71a-949c-5f22-98a6-75322fa13620',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'JEEP',
 'model': 'RENEGADE',
 'odometer': 37944,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_9846a71a-949c-5f22-98a6-75322fa13620/9846a71a-949c-5f22-98a6-75322fa13620_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 526188,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/575/addcb1c702614f949522fc5c38893a15/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2023-07-28',
 'seats_number': 5,
 'seller_name': None,
 'title': 'JEEP RENEGADE 4xe 1.3 PHEV 177 kW(240CV) Trailhawk AT',
 'transmission': 'manual',
 'trim': '4xe 1.3 PHEV 177 kW(240CV) Trailhawk AT',
 'vin': None,
 'year': 2023}
2025-12-05 20:00:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788750%27 HTTP/1.1" 200 None
2025-12-05 20:00:55 [ayvens.es] INFO: Saving data for 1788750: {'created_time': 1764964855.395736, 'last_price_update_time': 1764964855.395748, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788750') HTTP/1.1" 204 0
2025-12-05 20:00:55 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 786d7a25-8513-559f-bf2d-612205bd215c with identifier: 1788756
2025-12-05 20:00:55 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 786d7a25-8513-559f-bf2d-612205bd215c with identifier: 1788756
2025-12-05 20:00:55 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: 12 is greater than the maximum of 9

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['current_year_diff']:
    {'maximum': 9, 'type': 'number'}

On instance['current_year_diff']:
    12
2025-12-05 20:00:55 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: REGISTRATION_DATE_OLD failed for item: 786d7a25-8513-559f-bf2d-612205bd215c spider: ayvens.es
2025-12-05 20:00:55 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 786d7a25-8513-559f-bf2d-612205bd215c due to rule: registration_date_old.json, error: 12 is greater than the maximum of 9

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['current_year_diff']:
    {'maximum': 9, 'type': 'number'}

On instance['current_year_diff']:
    12
{'auction_title': '',
 'c_o_2_emission_value': 99,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '786d7a25-8513-559f-bf2d-612205bd215c',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'TOYOTA',
 'model': 'AYGO',
 'odometer': 17480,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_786d7a25-8513-559f-bf2d-612205bd215c/786d7a25-8513-559f-bf2d-612205bd215c_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 345399,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/809/18e3a6f9aaf64244b3cae0ff5e320441/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2013-03-01',
 'seats_number': 4,
 'seller_name': None,
 'title': 'TOYOTA AYGO 70 Live',
 'transmission': 'manual',
 'trim': '70 Live',
 'vin': None,
 'year': 2013}
2025-12-05 20:00:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788756%27 HTTP/1.1" 200 None
2025-12-05 20:00:55 [ayvens.es] INFO: Saving data for 1788756: {'created_time': 1764964855.685908, 'last_price_update_time': 1764964855.685918, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788756') HTTP/1.1" 204 0
2025-12-05 20:00:55 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/948/4a249ff5e3cc4671b2f0b04d4aa075ff/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:55 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/985/8498199a8dc54775a7092e8b4b60cb6a/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:55 [ayvens.es] INFO: Found listing with ID: 1788766
2025-12-05 20:00:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788766%27 HTTP/1.1" 200 None
2025-12-05 20:00:55 [ayvens.es] INFO: Scrape type for 1788766: 1
2025-12-05 20:00:55 [ayvens.es] INFO: Extracting data for car ID: 1788766
2025-12-05 20:00:55 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 76ba6998-78d9-593d-af5c-58d8c27c3951 with identifier: 1788766
2025-12-05 20:00:55 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 76ba6998-78d9-593d-af5c-58d8c27c3951 with scrape type 1
2025-12-05 20:00:55 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 76ba6998-78d9-593d-af5c-58d8c27c3951
2025-12-05 20:00:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F791%252Fa155b115f4a0447098f6d6f02032e7d4%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:55 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/791/a155b115f4a0447098f6d6f02032e7d4/images/large/1.jpg not downloaded yet
2025-12-05 20:00:55 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:55 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:55 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:55 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b05446d0>
2025-12-05 20:00:55 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:55 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:56 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:00:56 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:56 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:56 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:56 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_d0eb57d9-d3bf-5971-a676-f61bd60b7cae/d0eb57d9-d3bf-5971-a676-f61bd60b7cae_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:56 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:56 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F023%252F1ab4b01e62d34b93b3114cc22b6161a0%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:56 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/023/1ab4b01e62d34b93b3114cc22b6161a0/images/large/1.jpg saved to cache
2025-12-05 20:00:56 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item d0eb57d9-d3bf-5971-a676-f61bd60b7cae
2025-12-05 20:00:56 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: d0eb57d9-d3bf-5971-a676-f61bd60b7cae, skipping ID generation
2025-12-05 20:00:56 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item d0eb57d9-d3bf-5971-a676-f61bd60b7cae with scrape type 1
2025-12-05 20:00:56 [crawlers.middlewares.photo_download_middleware] INFO: Removing item d0eb57d9-d3bf-5971-a676-f61bd60b7cae from tracking dictionaries
2025-12-05 20:00:56 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item d0eb57d9-d3bf-5971-a676-f61bd60b7cae sending to next pipeline
2025-12-05 20:00:56 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: d0eb57d9-d3bf-5971-a676-f61bd60b7cae, identifier: 1788757
2025-12-05 20:00:56 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:56 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:56 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:56 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a946d0>
2025-12-05 20:00:56 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:56 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:56 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1969
2025-12-05 20:00:56 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:56 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:56 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:56 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_ddd41936-fe7a-5eea-b092-b77e5bf095c8/ddd41936-fe7a-5eea-b092-b77e5bf095c8_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:56 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:56 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F488%252F422327244e1c4d5b945c27b0e9cce062%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:56 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/488/422327244e1c4d5b945c27b0e9cce062/images/large/1.jpg saved to cache
2025-12-05 20:00:56 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item ddd41936-fe7a-5eea-b092-b77e5bf095c8
2025-12-05 20:00:56 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: ddd41936-fe7a-5eea-b092-b77e5bf095c8, skipping ID generation
2025-12-05 20:00:56 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ddd41936-fe7a-5eea-b092-b77e5bf095c8 with scrape type 1
2025-12-05 20:00:56 [crawlers.middlewares.photo_download_middleware] INFO: Removing item ddd41936-fe7a-5eea-b092-b77e5bf095c8 from tracking dictionaries
2025-12-05 20:00:56 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item ddd41936-fe7a-5eea-b092-b77e5bf095c8 sending to next pipeline
2025-12-05 20:00:56 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: ddd41936-fe7a-5eea-b092-b77e5bf095c8, identifier: 1788758
2025-12-05 20:00:56 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: d0eb57d9-d3bf-5971-a676-f61bd60b7cae with identifier: 1788757
2025-12-05 20:00:56 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: d0eb57d9-d3bf-5971-a676-f61bd60b7cae with identifier: 1788757
2025-12-05 20:00:56 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: d0eb57d9-d3bf-5971-a676-f61bd60b7cae with identifier: 1788757 to the API
2025-12-05 20:00:56 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:00:56 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:00:57 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/023/1ab4b01e62d34b93b3114cc22b6161a0/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 96,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'gas',
 'id': 'd0eb57d9-d3bf-5971-a676-f61bd60b7cae',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SKODA',
 'model': 'OCTAVIA',
 'odometer': 56990,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_d0eb57d9-d3bf-5971-a676-f61bd60b7cae/d0eb57d9-d3bf-5971-a676-f61bd60b7cae_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 347711,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/023/1ab4b01e62d34b93b3114cc22b6161a0/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-08-07',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SKODA OCTAVIA Combi 1.5 TGI 96kW(130CV) Ambit. CNG DSG',
 'transmission': 'manual',
 'trim': 'Combi 1.5 TGI 96kW(130CV) Ambit. CNG DSG',
 'vin': None,
 'year': 2020}
2025-12-05 20:00:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788757%27 HTTP/1.1" 200 None
2025-12-05 20:00:57 [ayvens.es] INFO: Saving data for 1788757: {'created_time': 1764964857.127696, 'last_price_update_time': 1764964857.127706, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788757') HTTP/1.1" 204 0
2025-12-05 20:00:57 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: ddd41936-fe7a-5eea-b092-b77e5bf095c8 with identifier: 1788758
2025-12-05 20:00:57 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: ddd41936-fe7a-5eea-b092-b77e5bf095c8 with identifier: 1788758
2025-12-05 20:00:57 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'peugeot', 'model': '2008', 'trim': 'active puretech 100 s&s bvm6', 'title': 'peugeot 2008 active puretech 100 s&s bvm6', 'price': 0, 'odometer': 78820, 'odometer_reading_unit': 'km', 'registration_date': '2020-11-10', 'year': 2020, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.2l', 'c_o_2_emission_value': 122, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'ddd41936-fe7a-5eea-b092-b77e5bf095c8', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ddd41936-fe7a-5eea-b092-b77e5bf095c8/ddd41936-fe7a-5eea-b092-b77e5bf095c8_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/488/422327244e1c4d5b945c27b0e9cce062/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 319129}], 'current_year_diff': 5, 'age_in_months': 60, 'make_model': 'peugeot 2008', 'sanitized_make': 'peugeot', 'odometer_in_km': 78820}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 60,
     'auction_title': '',
     'c_o_2_emission_value': 122,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.2l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': 'ddd41936-fe7a-5eea-b092-b77e5bf095c8',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'peugeot',
     'make_model': 'peugeot 2008',
     'model': '2008',
     'odometer': 78820,
     'odometer_in_km': 78820,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ddd41936-fe7a-5eea-b092-b77e5bf095c8/ddd41936-fe7a-5eea-b092-b77e5bf095c8_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 319129,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/488/422327244e1c4d5b945c27b0e9cce062/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-11-10',
     'sanitized_make': 'peugeot',
     'seats_number': 5,
     'seller_name': None,
     'title': 'peugeot 2008 active puretech 100 s&s bvm6',
     'transmission': 'manual',
     'trim': 'active puretech 100 s&s bvm6',
     'vin': None,
     'year': 2020}
2025-12-05 20:00:57 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: PURETECH_FOR_COUNTRY failed for item: ddd41936-fe7a-5eea-b092-b77e5bf095c8 spider: ayvens.es
2025-12-05 20:00:57 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: ddd41936-fe7a-5eea-b092-b77e5bf095c8 due to rule: puretech_for_country.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'peugeot', 'model': '2008', 'trim': 'active puretech 100 s&s bvm6', 'title': 'peugeot 2008 active puretech 100 s&s bvm6', 'price': 0, 'odometer': 78820, 'odometer_reading_unit': 'km', 'registration_date': '2020-11-10', 'year': 2020, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.2l', 'c_o_2_emission_value': 122, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'ddd41936-fe7a-5eea-b092-b77e5bf095c8', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ddd41936-fe7a-5eea-b092-b77e5bf095c8/ddd41936-fe7a-5eea-b092-b77e5bf095c8_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/488/422327244e1c4d5b945c27b0e9cce062/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 319129}], 'current_year_diff': 5, 'age_in_months': 60, 'make_model': 'peugeot 2008', 'sanitized_make': 'peugeot', 'odometer_in_km': 78820}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 60,
     'auction_title': '',
     'c_o_2_emission_value': 122,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.2l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': 'ddd41936-fe7a-5eea-b092-b77e5bf095c8',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'peugeot',
     'make_model': 'peugeot 2008',
     'model': '2008',
     'odometer': 78820,
     'odometer_in_km': 78820,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ddd41936-fe7a-5eea-b092-b77e5bf095c8/ddd41936-fe7a-5eea-b092-b77e5bf095c8_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 319129,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/488/422327244e1c4d5b945c27b0e9cce062/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-11-10',
     'sanitized_make': 'peugeot',
     'seats_number': 5,
     'seller_name': None,
     'title': 'peugeot 2008 active puretech 100 s&s bvm6',
     'transmission': 'manual',
     'trim': 'active puretech 100 s&s bvm6',
     'vin': None,
     'year': 2020}
{'auction_title': '',
 'c_o_2_emission_value': 122,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.2L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': 'ddd41936-fe7a-5eea-b092-b77e5bf095c8',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': '2008',
 'odometer': 78820,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ddd41936-fe7a-5eea-b092-b77e5bf095c8/ddd41936-fe7a-5eea-b092-b77e5bf095c8_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 319129,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/488/422327244e1c4d5b945c27b0e9cce062/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-11-10',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT 2008 Active Puretech 100 S&S BVM6',
 'transmission': 'manual',
 'trim': 'Active Puretech 100 S&S BVM6',
 'vin': None,
 'year': 2020}
2025-12-05 20:00:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788758%27 HTTP/1.1" 200 None
2025-12-05 20:00:57 [ayvens.es] INFO: Saving data for 1788758: {'created_time': 1764964857.402241, 'last_price_update_time': 1764964857.402253, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788758') HTTP/1.1" 204 0
2025-12-05 20:00:57 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/650/45d30630f4144a909d28b0a9c1da36d9/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:57 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/179/2d75fee09b0d4894aee9c28f24e11f1b/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:57 [ayvens.es] INFO: Found listing with ID: 1788767
2025-12-05 20:00:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788767%27 HTTP/1.1" 200 None
2025-12-05 20:00:57 [ayvens.es] INFO: Scrape type for 1788767: 1
2025-12-05 20:00:57 [ayvens.es] INFO: Extracting data for car ID: 1788767
2025-12-05 20:00:57 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 6f47b538-6b4b-5da8-9ab4-8d12645813eb with identifier: 1788767
2025-12-05 20:00:57 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6f47b538-6b4b-5da8-9ab4-8d12645813eb with scrape type 1
2025-12-05 20:00:57 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 6f47b538-6b4b-5da8-9ab4-8d12645813eb
2025-12-05 20:00:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F925%252F134fe554ffab4fa0964be6cf19253578%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:57 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/925/134fe554ffab4fa0964be6cf19253578/images/large/1.jpg not downloaded yet
2025-12-05 20:00:57 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:57 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:57 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:57 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b05de4d0>
2025-12-05 20:00:57 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:57 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:57 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:00:57 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:57 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:57 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:57 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_6b760c32-fa26-597d-80d0-9052a5aba827/6b760c32-fa26-597d-80d0-9052a5aba827_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F948%252F4a249ff5e3cc4671b2f0b04d4aa075ff%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:57 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/948/4a249ff5e3cc4671b2f0b04d4aa075ff/images/large/1.jpg saved to cache
2025-12-05 20:00:57 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 6b760c32-fa26-597d-80d0-9052a5aba827
2025-12-05 20:00:57 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 6b760c32-fa26-597d-80d0-9052a5aba827, skipping ID generation
2025-12-05 20:00:57 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6b760c32-fa26-597d-80d0-9052a5aba827 with scrape type 1
2025-12-05 20:00:57 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 6b760c32-fa26-597d-80d0-9052a5aba827 from tracking dictionaries
2025-12-05 20:00:57 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 6b760c32-fa26-597d-80d0-9052a5aba827 sending to next pipeline
2025-12-05 20:00:57 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 6b760c32-fa26-597d-80d0-9052a5aba827, identifier: 1788760
2025-12-05 20:00:57 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:57 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:57 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:57 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0d51090>
2025-12-05 20:00:57 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:57 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:58 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:00:58 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:58 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:58 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:58 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_dd61a0df-df8f-59a4-aa84-0b8946bab97e/dd61a0df-df8f-59a4-aa84-0b8946bab97e_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F985%252F8498199a8dc54775a7092e8b4b60cb6a%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:58 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/985/8498199a8dc54775a7092e8b4b60cb6a/images/large/1.jpg saved to cache
2025-12-05 20:00:58 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item dd61a0df-df8f-59a4-aa84-0b8946bab97e
2025-12-05 20:00:58 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: dd61a0df-df8f-59a4-aa84-0b8946bab97e, skipping ID generation
2025-12-05 20:00:58 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item dd61a0df-df8f-59a4-aa84-0b8946bab97e with scrape type 1
2025-12-05 20:00:58 [crawlers.middlewares.photo_download_middleware] INFO: Removing item dd61a0df-df8f-59a4-aa84-0b8946bab97e from tracking dictionaries
2025-12-05 20:00:58 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item dd61a0df-df8f-59a4-aa84-0b8946bab97e sending to next pipeline
2025-12-05 20:00:58 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: dd61a0df-df8f-59a4-aa84-0b8946bab97e, identifier: 1788759
2025-12-05 20:00:58 [ayvens.es] INFO: Found listing with ID: 1788768
2025-12-05 20:00:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788768%27 HTTP/1.1" 200 None
2025-12-05 20:00:58 [ayvens.es] INFO: Scrape type for 1788768: 1
2025-12-05 20:00:58 [ayvens.es] INFO: Extracting data for car ID: 1788768
2025-12-05 20:00:58 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 0ad028f1-c051-5d7f-91dd-fa17a9482b62 with identifier: 1788768
2025-12-05 20:00:58 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 0ad028f1-c051-5d7f-91dd-fa17a9482b62 with scrape type 1
2025-12-05 20:00:58 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 0ad028f1-c051-5d7f-91dd-fa17a9482b62
2025-12-05 20:00:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F680%252Fb72aadb95daf4e38b6ce715e50b7f4a4%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:58 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/680/b72aadb95daf4e38b6ce715e50b7f4a4/images/large/1.jpg not downloaded yet
2025-12-05 20:00:58 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 6b760c32-fa26-597d-80d0-9052a5aba827 with identifier: 1788760
2025-12-05 20:00:58 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 6b760c32-fa26-597d-80d0-9052a5aba827 with identifier: 1788760
2025-12-05 20:00:58 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 6b760c32-fa26-597d-80d0-9052a5aba827 with identifier: 1788760 to the API
2025-12-05 20:00:58 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:00:58 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:00:58 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/948/4a249ff5e3cc4671b2f0b04d4aa075ff/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 116,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '6b760c32-fa26-597d-80d0-9052a5aba827',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'CITROËN',
 'model': 'C3',
 'odometer': 116003,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_6b760c32-fa26-597d-80d0-9052a5aba827/6b760c32-fa26-597d-80d0-9052a5aba827_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 339886,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/948/4a249ff5e3cc4671b2f0b04d4aa075ff/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-03-02',
 'seats_number': 5,
 'seller_name': None,
 'title': 'CITROËN C3 BlueHDi 75KW (100CV) S&S Live Pack',
 'transmission': 'manual',
 'trim': 'BlueHDi 75KW (100CV) S&S Live Pack',
 'vin': None,
 'year': 2021}
2025-12-05 20:00:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788760%27 HTTP/1.1" 200 None
2025-12-05 20:00:58 [ayvens.es] INFO: Saving data for 1788760: {'created_time': 1764964858.844327, 'last_price_update_time': 1764964858.844336, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788760') HTTP/1.1" 204 0
2025-12-05 20:00:58 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: dd61a0df-df8f-59a4-aa84-0b8946bab97e with identifier: 1788759
2025-12-05 20:00:58 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:00:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:00:58 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: dd61a0df-df8f-59a4-aa84-0b8946bab97e with identifier: 1788759
2025-12-05 20:00:59 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'serie', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'peugeot', 'model': '2008', 'trim': 'active puretech 100 s&s bvm6', 'title': 'peugeot 2008 active puretech 100 s&s bvm6', 'price': 0, 'odometer': 79360, 'odometer_reading_unit': 'km', 'registration_date': '2020-12-11', 'year': 2020, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.2l', 'c_o_2_emission_value': 122, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'dd61a0df-df8f-59a4-aa84-0b8946bab97e', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_dd61a0df-df8f-59a4-aa84-0b8946bab97e/dd61a0df-df8f-59a4-aa84-0b8946bab97e_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/985/8498199a8dc54775a7092e8b4b60cb6a/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 351782}], 'current_year_diff': 5, 'age_in_months': 59, 'make_model': 'peugeot 2008', 'sanitized_make': 'peugeot', 'odometer_in_km': 79360}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 59,
     'auction_title': '',
     'c_o_2_emission_value': 122,
     'color': 'serie',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.2l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': 'dd61a0df-df8f-59a4-aa84-0b8946bab97e',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'peugeot',
     'make_model': 'peugeot 2008',
     'model': '2008',
     'odometer': 79360,
     'odometer_in_km': 79360,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_dd61a0df-df8f-59a4-aa84-0b8946bab97e/dd61a0df-df8f-59a4-aa84-0b8946bab97e_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 351782,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/985/8498199a8dc54775a7092e8b4b60cb6a/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-12-11',
     'sanitized_make': 'peugeot',
     'seats_number': 5,
     'seller_name': None,
     'title': 'peugeot 2008 active puretech 100 s&s bvm6',
     'transmission': 'manual',
     'trim': 'active puretech 100 s&s bvm6',
     'vin': None,
     'year': 2020}
2025-12-05 20:00:59 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: PURETECH_FOR_COUNTRY failed for item: dd61a0df-df8f-59a4-aa84-0b8946bab97e spider: ayvens.es
2025-12-05 20:00:59 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: dd61a0df-df8f-59a4-aa84-0b8946bab97e due to rule: puretech_for_country.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'serie', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'peugeot', 'model': '2008', 'trim': 'active puretech 100 s&s bvm6', 'title': 'peugeot 2008 active puretech 100 s&s bvm6', 'price': 0, 'odometer': 79360, 'odometer_reading_unit': 'km', 'registration_date': '2020-12-11', 'year': 2020, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.2l', 'c_o_2_emission_value': 122, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'dd61a0df-df8f-59a4-aa84-0b8946bab97e', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_dd61a0df-df8f-59a4-aa84-0b8946bab97e/dd61a0df-df8f-59a4-aa84-0b8946bab97e_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/985/8498199a8dc54775a7092e8b4b60cb6a/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 351782}], 'current_year_diff': 5, 'age_in_months': 59, 'make_model': 'peugeot 2008', 'sanitized_make': 'peugeot', 'odometer_in_km': 79360}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 59,
     'auction_title': '',
     'c_o_2_emission_value': 122,
     'color': 'serie',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.2l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': 'dd61a0df-df8f-59a4-aa84-0b8946bab97e',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'peugeot',
     'make_model': 'peugeot 2008',
     'model': '2008',
     'odometer': 79360,
     'odometer_in_km': 79360,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_dd61a0df-df8f-59a4-aa84-0b8946bab97e/dd61a0df-df8f-59a4-aa84-0b8946bab97e_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 351782,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/985/8498199a8dc54775a7092e8b4b60cb6a/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-12-11',
     'sanitized_make': 'peugeot',
     'seats_number': 5,
     'seller_name': None,
     'title': 'peugeot 2008 active puretech 100 s&s bvm6',
     'transmission': 'manual',
     'trim': 'active puretech 100 s&s bvm6',
     'vin': None,
     'year': 2020}
{'auction_title': '',
 'c_o_2_emission_value': 122,
 'color': 'Serie',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.2L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': 'dd61a0df-df8f-59a4-aa84-0b8946bab97e',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': '2008',
 'odometer': 79360,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_dd61a0df-df8f-59a4-aa84-0b8946bab97e/dd61a0df-df8f-59a4-aa84-0b8946bab97e_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 351782,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/985/8498199a8dc54775a7092e8b4b60cb6a/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-12-11',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT 2008 Active Puretech 100 S&S BVM6',
 'transmission': 'manual',
 'trim': 'Active Puretech 100 S&S BVM6',
 'vin': None,
 'year': 2020}
2025-12-05 20:00:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788759%27 HTTP/1.1" 200 None
2025-12-05 20:00:59 [ayvens.es] INFO: Saving data for 1788759: {'created_time': 1764964859.231476, 'last_price_update_time': 1764964859.231489, 'auction_closing_time': 1765281600.0}
2025-12-05 20:00:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788759') HTTP/1.1" 204 0
2025-12-05 20:00:59 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/678/7631af3845104ceca1c9aff6509a5c01/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:59 [ayvens.es] INFO: Found listing with ID: 1788769
2025-12-05 20:00:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788769%27 HTTP/1.1" 200 None
2025-12-05 20:00:59 [ayvens.es] INFO: Scrape type for 1788769: 1
2025-12-05 20:00:59 [ayvens.es] INFO: Extracting data for car ID: 1788769
2025-12-05 20:00:59 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 2eeab8fc-775f-576e-afd3-f9c463cddb58 with identifier: 1788769
2025-12-05 20:00:59 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 2eeab8fc-775f-576e-afd3-f9c463cddb58 with scrape type 1
2025-12-05 20:00:59 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 2eeab8fc-775f-576e-afd3-f9c463cddb58
2025-12-05 20:00:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F621%252Fa2676d2e155c45708aba00720fa3a65c%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:00:59 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/621/a2676d2e155c45708aba00720fa3a65c/images/large/1.jpg not downloaded yet
2025-12-05 20:00:59 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/974/7ce53bcd81cd4ef3bb838dd0adbb8c60/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:00:59 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:59 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:59 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:59 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0441190>
2025-12-05 20:00:59 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:59 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:00:59 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1969
2025-12-05 20:00:59 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:00:59 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:00:59 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:00:59 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_1067cb2b-6756-531f-b430-f5c2bf849c5a/1067cb2b-6756-531f-b430-f5c2bf849c5a_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:00:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:00:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F650%252F45d30630f4144a909d28b0a9c1da36d9%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:00:59 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/650/45d30630f4144a909d28b0a9c1da36d9/images/large/1.jpg saved to cache
2025-12-05 20:00:59 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 1067cb2b-6756-531f-b430-f5c2bf849c5a
2025-12-05 20:00:59 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 1067cb2b-6756-531f-b430-f5c2bf849c5a, skipping ID generation
2025-12-05 20:00:59 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 1067cb2b-6756-531f-b430-f5c2bf849c5a with scrape type 1
2025-12-05 20:00:59 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 1067cb2b-6756-531f-b430-f5c2bf849c5a from tracking dictionaries
2025-12-05 20:00:59 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 1067cb2b-6756-531f-b430-f5c2bf849c5a sending to next pipeline
2025-12-05 20:00:59 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 1067cb2b-6756-531f-b430-f5c2bf849c5a, identifier: 1788761
2025-12-05 20:00:59 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:00:59 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:00:59 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:00:59 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b051d290>
2025-12-05 20:00:59 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:00:59 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:00 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:01:00 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:00 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:00 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:00 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c/dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F179%252F2d75fee09b0d4894aee9c28f24e11f1b%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:00 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/179/2d75fee09b0d4894aee9c28f24e11f1b/images/large/1.jpg saved to cache
2025-12-05 20:01:00 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c
2025-12-05 20:01:00 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c, skipping ID generation
2025-12-05 20:01:00 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c with scrape type 1
2025-12-05 20:01:00 [crawlers.middlewares.photo_download_middleware] INFO: Removing item dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c from tracking dictionaries
2025-12-05 20:01:00 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c sending to next pipeline
2025-12-05 20:01:00 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c, identifier: 1788762
2025-12-05 20:01:00 [ayvens.es] INFO: Found listing with ID: 1788770
2025-12-05 20:01:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788770%27 HTTP/1.1" 200 None
2025-12-05 20:01:00 [ayvens.es] INFO: Scrape type for 1788770: 1
2025-12-05 20:01:00 [ayvens.es] INFO: Extracting data for car ID: 1788770
2025-12-05 20:01:00 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 562b2353-64f1-5e52-a46f-adb192edfff9 with identifier: 1788770
2025-12-05 20:01:00 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 562b2353-64f1-5e52-a46f-adb192edfff9 with scrape type 1
2025-12-05 20:01:00 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 562b2353-64f1-5e52-a46f-adb192edfff9
2025-12-05 20:01:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F760%252F2ebe9dd8d1104f18a9ea65f2f29739b5%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:00 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/760/2ebe9dd8d1104f18a9ea65f2f29739b5/images/large/1.jpg not downloaded yet
2025-12-05 20:01:00 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 1067cb2b-6756-531f-b430-f5c2bf849c5a with identifier: 1788761
2025-12-05 20:01:00 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:01:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:01:00 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 1067cb2b-6756-531f-b430-f5c2bf849c5a with identifier: 1788761
2025-12-05 20:01:00 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 1067cb2b-6756-531f-b430-f5c2bf849c5a with identifier: 1788761 to the API
2025-12-05 20:01:00 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:00 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:00 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/650/45d30630f4144a909d28b0a9c1da36d9/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 120,
 'color': 'Plata',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '1067cb2b-6756-531f-b430-f5c2bf849c5a',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': '508',
 'odometer': 128294,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_1067cb2b-6756-531f-b430-f5c2bf849c5a/1067cb2b-6756-531f-b430-f5c2bf849c5a_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 330238,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/650/45d30630f4144a909d28b0a9c1da36d9/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2022-07-20',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT 508 5P Active Pack BlueHDi 130 S&S EAT8',
 'transmission': 'manual',
 'trim': '5P Active Pack BlueHDi 130 S&S EAT8',
 'vin': None,
 'year': 2022}
2025-12-05 20:01:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788761%27 HTTP/1.1" 200 None
2025-12-05 20:01:00 [ayvens.es] INFO: Saving data for 1788761: {'created_time': 1764964860.727134, 'last_price_update_time': 1764964860.727146, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788761') HTTP/1.1" 204 0
2025-12-05 20:01:00 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c with identifier: 1788762
2025-12-05 20:01:00 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c with identifier: 1788762
2025-12-05 20:01:00 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'peugeot', 'model': '2008', 'trim': 'active puretech 100 s&s bvm6', 'title': 'peugeot 2008 active puretech 100 s&s bvm6', 'price': 0, 'odometer': 80999, 'odometer_reading_unit': 'km', 'registration_date': '2020-10-26', 'year': 2020, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.2l', 'c_o_2_emission_value': 122, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c/dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/179/2d75fee09b0d4894aee9c28f24e11f1b/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 282045}], 'current_year_diff': 5, 'age_in_months': 61, 'make_model': 'peugeot 2008', 'sanitized_make': 'peugeot', 'odometer_in_km': 80999}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 61,
     'auction_title': '',
     'c_o_2_emission_value': 122,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.2l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': 'dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'peugeot',
     'make_model': 'peugeot 2008',
     'model': '2008',
     'odometer': 80999,
     'odometer_in_km': 80999,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c/dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 282045,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/179/2d75fee09b0d4894aee9c28f24e11f1b/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-10-26',
     'sanitized_make': 'peugeot',
     'seats_number': 5,
     'seller_name': None,
     'title': 'peugeot 2008 active puretech 100 s&s bvm6',
     'transmission': 'manual',
     'trim': 'active puretech 100 s&s bvm6',
     'vin': None,
     'year': 2020}
2025-12-05 20:01:00 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: PURETECH_FOR_COUNTRY failed for item: dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c spider: ayvens.es
2025-12-05 20:01:01 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c due to rule: puretech_for_country.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'peugeot', 'model': '2008', 'trim': 'active puretech 100 s&s bvm6', 'title': 'peugeot 2008 active puretech 100 s&s bvm6', 'price': 0, 'odometer': 80999, 'odometer_reading_unit': 'km', 'registration_date': '2020-10-26', 'year': 2020, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.2l', 'c_o_2_emission_value': 122, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c/dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/179/2d75fee09b0d4894aee9c28f24e11f1b/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 282045}], 'current_year_diff': 5, 'age_in_months': 61, 'make_model': 'peugeot 2008', 'sanitized_make': 'peugeot', 'odometer_in_km': 80999}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 61,
     'auction_title': '',
     'c_o_2_emission_value': 122,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.2l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': 'dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'peugeot',
     'make_model': 'peugeot 2008',
     'model': '2008',
     'odometer': 80999,
     'odometer_in_km': 80999,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c/dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 282045,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/179/2d75fee09b0d4894aee9c28f24e11f1b/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-10-26',
     'sanitized_make': 'peugeot',
     'seats_number': 5,
     'seller_name': None,
     'title': 'peugeot 2008 active puretech 100 s&s bvm6',
     'transmission': 'manual',
     'trim': 'active puretech 100 s&s bvm6',
     'vin': None,
     'year': 2020}
{'auction_title': '',
 'c_o_2_emission_value': 122,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.2L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': 'dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': '2008',
 'odometer': 80999,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c/dcc3bd01-e16a-58f8-9c35-33e5b0a8ee0c_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 282045,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/179/2d75fee09b0d4894aee9c28f24e11f1b/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-10-26',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT 2008 Active Puretech 100 S&S BVM6',
 'transmission': 'manual',
 'trim': 'Active Puretech 100 S&S BVM6',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788762%27 HTTP/1.1" 200 None
2025-12-05 20:01:01 [ayvens.es] INFO: Saving data for 1788762: {'created_time': 1764964861.033934, 'last_price_update_time': 1764964861.03397, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788762') HTTP/1.1" 204 0
2025-12-05 20:01:01 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/774/a4ce057bc3df4fd5a4e74db182dac929/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:01 [ayvens.es] INFO: Found listing with ID: 1788771
2025-12-05 20:01:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788771%27 HTTP/1.1" 200 None
2025-12-05 20:01:01 [ayvens.es] INFO: Scrape type for 1788771: 1
2025-12-05 20:01:01 [ayvens.es] INFO: Extracting data for car ID: 1788771
2025-12-05 20:01:01 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: e0fd7b43-fbd3-5d03-a00c-671ccc573ab4 with identifier: 1788771
2025-12-05 20:01:01 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e0fd7b43-fbd3-5d03-a00c-671ccc573ab4 with scrape type 1
2025-12-05 20:01:01 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item e0fd7b43-fbd3-5d03-a00c-671ccc573ab4
2025-12-05 20:01:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F536%252F7ce0d532ce5940239ed7784fa7abb61a%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:01 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/536/7ce0d532ce5940239ed7784fa7abb61a/images/large/1.jpg not downloaded yet
2025-12-05 20:01:01 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/791/a155b115f4a0447098f6d6f02032e7d4/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:01 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:01 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:01 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:01 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a40a50>
2025-12-05 20:01:01 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:01 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:01 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:01 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:01 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:01 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:01 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_88a958bc-2914-56f4-b2a3-b4a183a78bda/88a958bc-2914-56f4-b2a3-b4a183a78bda_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F678%252F7631af3845104ceca1c9aff6509a5c01%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:01 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/678/7631af3845104ceca1c9aff6509a5c01/images/large/1.jpg saved to cache
2025-12-05 20:01:01 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 88a958bc-2914-56f4-b2a3-b4a183a78bda
2025-12-05 20:01:01 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 88a958bc-2914-56f4-b2a3-b4a183a78bda, skipping ID generation
2025-12-05 20:01:01 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 88a958bc-2914-56f4-b2a3-b4a183a78bda with scrape type 1
2025-12-05 20:01:01 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 88a958bc-2914-56f4-b2a3-b4a183a78bda from tracking dictionaries
2025-12-05 20:01:01 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 88a958bc-2914-56f4-b2a3-b4a183a78bda sending to next pipeline
2025-12-05 20:01:01 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 88a958bc-2914-56f4-b2a3-b4a183a78bda, identifier: 1788764
2025-12-05 20:01:01 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:01 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:01 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:01 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a36490>
2025-12-05 20:01:01 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:01 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:01 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:01:01 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:02 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:02 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:02 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4/9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F974%252F7ce53bcd81cd4ef3bb838dd0adbb8c60%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:02 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/974/7ce53bcd81cd4ef3bb838dd0adbb8c60/images/large/1.jpg saved to cache
2025-12-05 20:01:02 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4
2025-12-05 20:01:02 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4, skipping ID generation
2025-12-05 20:01:02 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4 with scrape type 1
2025-12-05 20:01:02 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4 from tracking dictionaries
2025-12-05 20:01:02 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4 sending to next pipeline
2025-12-05 20:01:02 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4, identifier: 1788763
2025-12-05 20:01:02 [ayvens.es] INFO: Found listing with ID: 1788772
2025-12-05 20:01:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788772%27 HTTP/1.1" 200 None
2025-12-05 20:01:02 [ayvens.es] INFO: Scrape type for 1788772: 1
2025-12-05 20:01:02 [ayvens.es] INFO: Extracting data for car ID: 1788772
2025-12-05 20:01:02 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 20fe41fd-d76a-565f-b590-2dd0feb845f0 with identifier: 1788772
2025-12-05 20:01:02 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 20fe41fd-d76a-565f-b590-2dd0feb845f0 with scrape type 1
2025-12-05 20:01:02 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 20fe41fd-d76a-565f-b590-2dd0feb845f0
2025-12-05 20:01:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F267%252Fbbe6679ac0ab4398a62617fe218b8203%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:02 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/267/bbe6679ac0ab4398a62617fe218b8203/images/large/1.jpg not downloaded yet
2025-12-05 20:01:02 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 88a958bc-2914-56f4-b2a3-b4a183a78bda with identifier: 1788764
2025-12-05 20:01:02 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:01:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:01:02 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 88a958bc-2914-56f4-b2a3-b4a183a78bda with identifier: 1788764
2025-12-05 20:01:02 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 88a958bc-2914-56f4-b2a3-b4a183a78bda with identifier: 1788764 to the API
2025-12-05 20:01:02 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:02 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:02 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/678/7631af3845104ceca1c9aff6509a5c01/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 109,
 'color': 'Glaciar',
 'currency': 'EUR',
 'doors_number': 3,
 'engine': '0.9L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'gas',
 'id': '88a958bc-2914-56f4-b2a3-b4a183a78bda',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'CLIO',
 'odometer': 89135,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_88a958bc-2914-56f4-b2a3-b4a183a78bda/88a958bc-2914-56f4-b2a3-b4a183a78bda_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 317092,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/678/7631af3845104ceca1c9aff6509a5c01/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-07-05',
 'seats_number': 2,
 'seller_name': None,
 'title': 'RENAULT CLIO (O) Societe Energy TCe 66 kW (90 CV) GLP -18',
 'transmission': 'manual',
 'trim': '(O) Societe Energy TCe 66 kW (90 CV) GLP -18',
 'vin': None,
 'year': 2019}
2025-12-05 20:01:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788764%27 HTTP/1.1" 200 None
2025-12-05 20:01:02 [ayvens.es] INFO: Saving data for 1788764: {'created_time': 1764964862.731938, 'last_price_update_time': 1764964862.731949, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788764') HTTP/1.1" 204 0
2025-12-05 20:01:02 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4 with identifier: 1788763
2025-12-05 20:01:02 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4 with identifier: 1788763
2025-12-05 20:01:02 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'peugeot', 'model': '2008', 'trim': 'active puretech 100 s&s bvm6', 'title': 'peugeot 2008 active puretech 100 s&s bvm6', 'price': 0, 'odometer': 79368, 'odometer_reading_unit': 'km', 'registration_date': '2020-12-16', 'year': 2020, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.2l', 'c_o_2_emission_value': 122, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4/9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/974/7ce53bcd81cd4ef3bb838dd0adbb8c60/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 343428}], 'current_year_diff': 5, 'age_in_months': 59, 'make_model': 'peugeot 2008', 'sanitized_make': 'peugeot', 'odometer_in_km': 79368}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 59,
     'auction_title': '',
     'c_o_2_emission_value': 122,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.2l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': '9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'peugeot',
     'make_model': 'peugeot 2008',
     'model': '2008',
     'odometer': 79368,
     'odometer_in_km': 79368,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4/9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 343428,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/974/7ce53bcd81cd4ef3bb838dd0adbb8c60/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-12-16',
     'sanitized_make': 'peugeot',
     'seats_number': 5,
     'seller_name': None,
     'title': 'peugeot 2008 active puretech 100 s&s bvm6',
     'transmission': 'manual',
     'trim': 'active puretech 100 s&s bvm6',
     'vin': None,
     'year': 2020}
2025-12-05 20:01:02 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: PURETECH_FOR_COUNTRY failed for item: 9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4 spider: ayvens.es
2025-12-05 20:01:02 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4 due to rule: puretech_for_country.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'peugeot', 'model': '2008', 'trim': 'active puretech 100 s&s bvm6', 'title': 'peugeot 2008 active puretech 100 s&s bvm6', 'price': 0, 'odometer': 79368, 'odometer_reading_unit': 'km', 'registration_date': '2020-12-16', 'year': 2020, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.2l', 'c_o_2_emission_value': 122, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4/9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/974/7ce53bcd81cd4ef3bb838dd0adbb8c60/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 343428}], 'current_year_diff': 5, 'age_in_months': 59, 'make_model': 'peugeot 2008', 'sanitized_make': 'peugeot', 'odometer_in_km': 79368}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 59,
     'auction_title': '',
     'c_o_2_emission_value': 122,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.2l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': '9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'peugeot',
     'make_model': 'peugeot 2008',
     'model': '2008',
     'odometer': 79368,
     'odometer_in_km': 79368,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4/9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 343428,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/974/7ce53bcd81cd4ef3bb838dd0adbb8c60/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-12-16',
     'sanitized_make': 'peugeot',
     'seats_number': 5,
     'seller_name': None,
     'title': 'peugeot 2008 active puretech 100 s&s bvm6',
     'transmission': 'manual',
     'trim': 'active puretech 100 s&s bvm6',
     'vin': None,
     'year': 2020}
{'auction_title': '',
 'c_o_2_emission_value': 122,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.2L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': '2008',
 'odometer': 79368,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4/9d2be1b8-e5fe-5fb2-89f1-c5569fbdfea4_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 343428,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/974/7ce53bcd81cd4ef3bb838dd0adbb8c60/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-12-16',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT 2008 Active Puretech 100 S&S BVM6',
 'transmission': 'manual',
 'trim': 'Active Puretech 100 S&S BVM6',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788763%27 HTTP/1.1" 200 None
2025-12-05 20:01:03 [ayvens.es] INFO: Saving data for 1788763: {'created_time': 1764964863.004254, 'last_price_update_time': 1764964863.004266, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788763') HTTP/1.1" 204 0
2025-12-05 20:01:03 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/925/134fe554ffab4fa0964be6cf19253578/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:03 [ayvens.es] INFO: Found listing with ID: 1788773
2025-12-05 20:01:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788773%27 HTTP/1.1" 200 None
2025-12-05 20:01:03 [ayvens.es] INFO: Scrape type for 1788773: 1
2025-12-05 20:01:03 [ayvens.es] INFO: Extracting data for car ID: 1788773
2025-12-05 20:01:03 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 5e68e2ec-0c20-565b-b28e-711cc82caee0 with identifier: 1788773
2025-12-05 20:01:03 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 5e68e2ec-0c20-565b-b28e-711cc82caee0 with scrape type 1
2025-12-05 20:01:03 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 5e68e2ec-0c20-565b-b28e-711cc82caee0
2025-12-05 20:01:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F193%252F721deb3d172045179808e8c1b1dd00a9%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:03 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/193/721deb3d172045179808e8c1b1dd00a9/images/large/1.jpg not downloaded yet
2025-12-05 20:01:03 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/680/b72aadb95daf4e38b6ce715e50b7f4a4/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:03 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:03 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:03 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:03 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b4e850>
2025-12-05 20:01:03 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:03 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:03 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:03 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:03 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:03 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:03 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_52f27cf9-37d2-5078-9c54-800687b9a9a5/52f27cf9-37d2-5078-9c54-800687b9a9a5_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F774%252Fa4ce057bc3df4fd5a4e74db182dac929%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:03 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/774/a4ce057bc3df4fd5a4e74db182dac929/images/large/1.jpg saved to cache
2025-12-05 20:01:03 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 52f27cf9-37d2-5078-9c54-800687b9a9a5
2025-12-05 20:01:03 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 52f27cf9-37d2-5078-9c54-800687b9a9a5, skipping ID generation
2025-12-05 20:01:03 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 52f27cf9-37d2-5078-9c54-800687b9a9a5 with scrape type 1
2025-12-05 20:01:03 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 52f27cf9-37d2-5078-9c54-800687b9a9a5 from tracking dictionaries
2025-12-05 20:01:03 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 52f27cf9-37d2-5078-9c54-800687b9a9a5 sending to next pipeline
2025-12-05 20:01:03 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 52f27cf9-37d2-5078-9c54-800687b9a9a5, identifier: 1788765
2025-12-05 20:01:03 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:03 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:03 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:03 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b054c550>
2025-12-05 20:01:03 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:03 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:03 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:03 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:03 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:03 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:03 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_76ba6998-78d9-593d-af5c-58d8c27c3951/76ba6998-78d9-593d-af5c-58d8c27c3951_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F791%252Fa155b115f4a0447098f6d6f02032e7d4%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:03 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/791/a155b115f4a0447098f6d6f02032e7d4/images/large/1.jpg saved to cache
2025-12-05 20:01:03 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 76ba6998-78d9-593d-af5c-58d8c27c3951
2025-12-05 20:01:03 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 76ba6998-78d9-593d-af5c-58d8c27c3951, skipping ID generation
2025-12-05 20:01:03 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 76ba6998-78d9-593d-af5c-58d8c27c3951 with scrape type 1
2025-12-05 20:01:03 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 76ba6998-78d9-593d-af5c-58d8c27c3951 from tracking dictionaries
2025-12-05 20:01:03 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 76ba6998-78d9-593d-af5c-58d8c27c3951 sending to next pipeline
2025-12-05 20:01:04 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 76ba6998-78d9-593d-af5c-58d8c27c3951, identifier: 1788766
2025-12-05 20:01:04 [ayvens.es] INFO: Found listing with ID: 1788774
2025-12-05 20:01:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788774%27 HTTP/1.1" 200 None
2025-12-05 20:01:04 [ayvens.es] INFO: Scrape type for 1788774: 1
2025-12-05 20:01:04 [ayvens.es] INFO: Extracting data for car ID: 1788774
2025-12-05 20:01:04 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: cc08d996-1736-599d-9047-d85adec3cc5a with identifier: 1788774
2025-12-05 20:01:04 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item cc08d996-1736-599d-9047-d85adec3cc5a with scrape type 1
2025-12-05 20:01:04 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item cc08d996-1736-599d-9047-d85adec3cc5a
2025-12-05 20:01:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F870%252F40d22532ee614c4e9cc068484630ee00%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:04 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/870/40d22532ee614c4e9cc068484630ee00/images/large/1.jpg not downloaded yet
2025-12-05 20:01:04 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 52f27cf9-37d2-5078-9c54-800687b9a9a5 with identifier: 1788765
2025-12-05 20:01:04 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 52f27cf9-37d2-5078-9c54-800687b9a9a5 with identifier: 1788765
2025-12-05 20:01:04 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'volkswagen' containing 'volkswagen' and model 'caddy' containing 'caddy'
2025-12-05 20:01:04 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 52f27cf9-37d2-5078-9c54-800687b9a9a5 spider: ayvens.es
2025-12-05 20:01:04 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 52f27cf9-37d2-5078-9c54-800687b9a9a5 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'volkswagen' containing 'volkswagen' and model 'caddy' containing 'caddy'
{'auction_title': '',
 'c_o_2_emission_value': 160,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '52f27cf9-37d2-5078-9c54-800687b9a9a5',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'CADDY',
 'odometer': 149647,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_52f27cf9-37d2-5078-9c54-800687b9a9a5/52f27cf9-37d2-5078-9c54-800687b9a9a5_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 337226,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/774/a4ce057bc3df4fd5a4e74db182dac929/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-11-30',
 'seats_number': 2,
 'seller_name': None,
 'title': 'VOLKSWAGEN CADDY Profes Furgon 2.0 TDI 90kW BMT 4Motion',
 'transmission': 'manual',
 'trim': 'Profes Furgon 2.0 TDI 90kW BMT 4Motion',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788765%27 HTTP/1.1" 200 None
2025-12-05 20:01:04 [ayvens.es] INFO: Saving data for 1788765: {'created_time': 1764964864.318497, 'last_price_update_time': 1764964864.31851, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788765') HTTP/1.1" 204 0
2025-12-05 20:01:04 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 76ba6998-78d9-593d-af5c-58d8c27c3951 with identifier: 1788766
2025-12-05 20:01:04 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 76ba6998-78d9-593d-af5c-58d8c27c3951 with identifier: 1788766
2025-12-05 20:01:04 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'peugeot' containing 'peugeot' and model 'expert' containing 'expert'
2025-12-05 20:01:04 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 76ba6998-78d9-593d-af5c-58d8c27c3951 spider: ayvens.es
2025-12-05 20:01:04 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 76ba6998-78d9-593d-af5c-58d8c27c3951 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'peugeot' containing 'peugeot' and model 'expert' containing 'expert'
{'auction_title': '',
 'c_o_2_emission_value': 142,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '76ba6998-78d9-593d-af5c-58d8c27c3951',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': 'EXPERT',
 'odometer': 108902,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_76ba6998-78d9-593d-af5c-58d8c27c3951/76ba6998-78d9-593d-af5c-58d8c27c3951_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 327558,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/791/a155b115f4a0447098f6d6f02032e7d4/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-05-26',
 'seats_number': 2,
 'seller_name': None,
 'title': 'PEUGEOT EXPERT Furgon 4x4 Dangel 1.5 BlueHDi 120 Stdard',
 'transmission': 'manual',
 'trim': 'Furgon 4x4 Dangel 1.5 BlueHDi 120 Stdard',
 'vin': None,
 'year': 2021}
2025-12-05 20:01:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788766%27 HTTP/1.1" 200 None
2025-12-05 20:01:04 [ayvens.es] INFO: Saving data for 1788766: {'created_time': 1764964864.597561, 'last_price_update_time': 1764964864.597571, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788766') HTTP/1.1" 204 0
2025-12-05 20:01:04 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/621/a2676d2e155c45708aba00720fa3a65c/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:04 [ayvens.es] INFO: Found listing with ID: 1788775
2025-12-05 20:01:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788775%27 HTTP/1.1" 200 None
2025-12-05 20:01:04 [ayvens.es] INFO: Scrape type for 1788775: 1
2025-12-05 20:01:04 [ayvens.es] INFO: Extracting data for car ID: 1788775
2025-12-05 20:01:04 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 374218cf-74ac-5703-9e62-bb769f58be63 with identifier: 1788775
2025-12-05 20:01:04 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 374218cf-74ac-5703-9e62-bb769f58be63 with scrape type 1
2025-12-05 20:01:04 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 374218cf-74ac-5703-9e62-bb769f58be63
2025-12-05 20:01:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F574%252Ff91e81c4443b460eb54886f5fd6be60d%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:04 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/574/f91e81c4443b460eb54886f5fd6be60d/images/large/1.jpg not downloaded yet
2025-12-05 20:01:04 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:04 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:04 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:04 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b04502d0>
2025-12-05 20:01:04 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:04 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:04 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:04 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:04 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:04 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:04 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_6f47b538-6b4b-5da8-9ab4-8d12645813eb/6f47b538-6b4b-5da8-9ab4-8d12645813eb_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F925%252F134fe554ffab4fa0964be6cf19253578%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:04 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/925/134fe554ffab4fa0964be6cf19253578/images/large/1.jpg saved to cache
2025-12-05 20:01:04 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 6f47b538-6b4b-5da8-9ab4-8d12645813eb
2025-12-05 20:01:04 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 6f47b538-6b4b-5da8-9ab4-8d12645813eb, skipping ID generation
2025-12-05 20:01:04 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6f47b538-6b4b-5da8-9ab4-8d12645813eb with scrape type 1
2025-12-05 20:01:04 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 6f47b538-6b4b-5da8-9ab4-8d12645813eb from tracking dictionaries
2025-12-05 20:01:04 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 6f47b538-6b4b-5da8-9ab4-8d12645813eb sending to next pipeline
2025-12-05 20:01:05 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 6f47b538-6b4b-5da8-9ab4-8d12645813eb, identifier: 1788767
2025-12-05 20:01:05 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:05 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:05 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:05 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0453910>
2025-12-05 20:01:05 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:05 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:05 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:05 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:05 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:05 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:05 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_0ad028f1-c051-5d7f-91dd-fa17a9482b62/0ad028f1-c051-5d7f-91dd-fa17a9482b62_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F680%252Fb72aadb95daf4e38b6ce715e50b7f4a4%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:05 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/680/b72aadb95daf4e38b6ce715e50b7f4a4/images/large/1.jpg saved to cache
2025-12-05 20:01:05 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 0ad028f1-c051-5d7f-91dd-fa17a9482b62
2025-12-05 20:01:05 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 0ad028f1-c051-5d7f-91dd-fa17a9482b62, skipping ID generation
2025-12-05 20:01:05 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 0ad028f1-c051-5d7f-91dd-fa17a9482b62 with scrape type 1
2025-12-05 20:01:05 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 0ad028f1-c051-5d7f-91dd-fa17a9482b62 from tracking dictionaries
2025-12-05 20:01:05 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 0ad028f1-c051-5d7f-91dd-fa17a9482b62 sending to next pipeline
2025-12-05 20:01:05 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 0ad028f1-c051-5d7f-91dd-fa17a9482b62, identifier: 1788768
2025-12-05 20:01:05 [ayvens.es] INFO: Found listing with ID: 1788776
2025-12-05 20:01:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788776%27 HTTP/1.1" 200 None
2025-12-05 20:01:05 [ayvens.es] INFO: Scrape type for 1788776: 1
2025-12-05 20:01:05 [ayvens.es] INFO: Extracting data for car ID: 1788776
2025-12-05 20:01:05 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 06434433-7dd4-50df-9d22-108f90d4c411 with identifier: 1788776
2025-12-05 20:01:05 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 06434433-7dd4-50df-9d22-108f90d4c411 with scrape type 1
2025-12-05 20:01:05 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 06434433-7dd4-50df-9d22-108f90d4c411
2025-12-05 20:01:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F525%252F24b2fea0f5b840d2abb5c43b6e60bfba%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:05 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/525/24b2fea0f5b840d2abb5c43b6e60bfba/images/large/1.jpg not downloaded yet
2025-12-05 20:01:05 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 6f47b538-6b4b-5da8-9ab4-8d12645813eb with identifier: 1788767
2025-12-05 20:01:05 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 6f47b538-6b4b-5da8-9ab4-8d12645813eb with identifier: 1788767
2025-12-05 20:01:05 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'toyota' containing 'toyota' and model 'hilux' containing 'hilux'
2025-12-05 20:01:05 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 6f47b538-6b4b-5da8-9ab4-8d12645813eb spider: ayvens.es
2025-12-05 20:01:06 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 6f47b538-6b4b-5da8-9ab4-8d12645813eb due to rule: makes_models_not_allowed.json, error: Rejected due to make 'toyota' containing 'toyota' and model 'hilux' containing 'hilux'
{'auction_title': '',
 'c_o_2_emission_value': 187,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 2,
 'engine': '2.4L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '6f47b538-6b4b-5da8-9ab4-8d12645813eb',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'TOYOTA',
 'model': 'HILUX',
 'odometer': 60068,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_6f47b538-6b4b-5da8-9ab4-8d12645813eb/6f47b538-6b4b-5da8-9ab4-8d12645813eb_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 352306,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/925/134fe554ffab4fa0964be6cf19253578/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-04-09',
 'seats_number': 2,
 'seller_name': None,
 'title': 'TOYOTA HILUX 2.4 D-4D Cabina Sencilla GX',
 'transmission': 'manual',
 'trim': '2.4 D-4D Cabina Sencilla GX',
 'vin': None,
 'year': 2019}
2025-12-05 20:01:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788767%27 HTTP/1.1" 200 None
2025-12-05 20:01:06 [ayvens.es] INFO: Saving data for 1788767: {'created_time': 1764964866.023062, 'last_price_update_time': 1764964866.02307, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788767') HTTP/1.1" 204 0
2025-12-05 20:01:06 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 0ad028f1-c051-5d7f-91dd-fa17a9482b62 with identifier: 1788768
2025-12-05 20:01:06 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 0ad028f1-c051-5d7f-91dd-fa17a9482b62 with identifier: 1788768
2025-12-05 20:01:06 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'volkswagen' containing 'volkswagen' and model 'caddy' containing 'caddy'
2025-12-05 20:01:06 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 0ad028f1-c051-5d7f-91dd-fa17a9482b62 spider: ayvens.es
2025-12-05 20:01:06 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 0ad028f1-c051-5d7f-91dd-fa17a9482b62 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'volkswagen' containing 'volkswagen' and model 'caddy' containing 'caddy'
{'auction_title': '',
 'c_o_2_emission_value': 127,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.4L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'gas',
 'id': '0ad028f1-c051-5d7f-91dd-fa17a9482b62',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'CADDY',
 'odometer': 103598,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_0ad028f1-c051-5d7f-91dd-fa17a9482b62/0ad028f1-c051-5d7f-91dd-fa17a9482b62_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 330457,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/680/b72aadb95daf4e38b6ce715e50b7f4a4/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-06-19',
 'seats_number': 2,
 'seller_name': None,
 'title': 'VOLKSWAGEN CADDY Profesional Furgon 1.4 TGI 81kW BM',
 'transmission': 'manual',
 'trim': 'Profesional Furgon 1.4 TGI 81kW BM',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788768%27 HTTP/1.1" 200 None
2025-12-05 20:01:06 [ayvens.es] INFO: Saving data for 1788768: {'created_time': 1764964866.2926, 'last_price_update_time': 1764964866.29261, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788768') HTTP/1.1" 204 0
2025-12-05 20:01:06 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/536/7ce0d532ce5940239ed7784fa7abb61a/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:06 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/760/2ebe9dd8d1104f18a9ea65f2f29739b5/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:06 [ayvens.es] INFO: Found listing with ID: 1788777
2025-12-05 20:01:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788777%27 HTTP/1.1" 200 None
2025-12-05 20:01:06 [ayvens.es] INFO: Scrape type for 1788777: 1
2025-12-05 20:01:06 [ayvens.es] INFO: Extracting data for car ID: 1788777
2025-12-05 20:01:06 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: f6efab77-d6e4-549d-8cf8-ecc622dde6fe with identifier: 1788777
2025-12-05 20:01:06 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f6efab77-d6e4-549d-8cf8-ecc622dde6fe with scrape type 1
2025-12-05 20:01:06 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item f6efab77-d6e4-549d-8cf8-ecc622dde6fe
2025-12-05 20:01:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F972%252F294a756b02b54c7089ad2664405d7098%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:06 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/972/294a756b02b54c7089ad2664405d7098/images/large/1.jpg not downloaded yet
2025-12-05 20:01:06 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/267/bbe6679ac0ab4398a62617fe218b8203/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:06 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:06 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:06 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:06 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0486690>
2025-12-05 20:01:06 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:06 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:06 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:01:06 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:06 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:06 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:06 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_2eeab8fc-775f-576e-afd3-f9c463cddb58/2eeab8fc-775f-576e-afd3-f9c463cddb58_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F621%252Fa2676d2e155c45708aba00720fa3a65c%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:06 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/621/a2676d2e155c45708aba00720fa3a65c/images/large/1.jpg saved to cache
2025-12-05 20:01:06 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 2eeab8fc-775f-576e-afd3-f9c463cddb58
2025-12-05 20:01:06 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 2eeab8fc-775f-576e-afd3-f9c463cddb58, skipping ID generation
2025-12-05 20:01:06 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 2eeab8fc-775f-576e-afd3-f9c463cddb58 with scrape type 1
2025-12-05 20:01:06 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 2eeab8fc-775f-576e-afd3-f9c463cddb58 from tracking dictionaries
2025-12-05 20:01:06 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 2eeab8fc-775f-576e-afd3-f9c463cddb58 sending to next pipeline
2025-12-05 20:01:06 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 2eeab8fc-775f-576e-afd3-f9c463cddb58, identifier: 1788769
2025-12-05 20:01:06 [ayvens.es] INFO: Found listing with ID: 1788778
2025-12-05 20:01:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788778%27 HTTP/1.1" 200 None
2025-12-05 20:01:06 [ayvens.es] INFO: Scrape type for 1788778: 1
2025-12-05 20:01:06 [ayvens.es] INFO: Extracting data for car ID: 1788778
2025-12-05 20:01:06 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 9c269e51-8b6e-56ed-8498-36d194208f53 with identifier: 1788778
2025-12-05 20:01:06 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9c269e51-8b6e-56ed-8498-36d194208f53 with scrape type 1
2025-12-05 20:01:06 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 9c269e51-8b6e-56ed-8498-36d194208f53
2025-12-05 20:01:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F397%252Fd576a5b8b43546fbae5ba93d4f8cf27c%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:06 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/397/d576a5b8b43546fbae5ba93d4f8cf27c/images/large/1.jpg not downloaded yet
2025-12-05 20:01:06 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 2eeab8fc-775f-576e-afd3-f9c463cddb58 with identifier: 1788769
2025-12-05 20:01:06 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 2eeab8fc-775f-576e-afd3-f9c463cddb58 with identifier: 1788769
2025-12-05 20:01:07 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 2eeab8fc-775f-576e-afd3-f9c463cddb58 with identifier: 1788769 to the API
2025-12-05 20:01:07 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:07 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:07 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/621/a2676d2e155c45708aba00720fa3a65c/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 96,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'gas',
 'id': '2eeab8fc-775f-576e-afd3-f9c463cddb58',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SKODA',
 'model': 'OCTAVIA',
 'odometer': 68825,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_2eeab8fc-775f-576e-afd3-f9c463cddb58/2eeab8fc-775f-576e-afd3-f9c463cddb58_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 358846,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/621/a2676d2e155c45708aba00720fa3a65c/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-08-24',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SKODA OCTAVIA Combi 1.5 TGI 96kW(130CV) Ambit. CNG DSG',
 'transmission': 'manual',
 'trim': 'Combi 1.5 TGI 96kW(130CV) Ambit. CNG DSG',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788769%27 HTTP/1.1" 200 None
2025-12-05 20:01:07 [ayvens.es] INFO: Saving data for 1788769: {'created_time': 1764964867.24294, 'last_price_update_time': 1764964867.242953, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788769') HTTP/1.1" 204 0
2025-12-05 20:01:07 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/193/721deb3d172045179808e8c1b1dd00a9/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:07 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/870/40d22532ee614c4e9cc068484630ee00/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:07 [ayvens.es] INFO: Found listing with ID: 1788779
2025-12-05 20:01:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788779%27 HTTP/1.1" 200 None
2025-12-05 20:01:07 [ayvens.es] INFO: Scrape type for 1788779: 1
2025-12-05 20:01:07 [ayvens.es] INFO: Extracting data for car ID: 1788779
2025-12-05 20:01:07 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 130d1d6b-df21-5081-a28d-4d4adc17a5d6 with identifier: 1788779
2025-12-05 20:01:07 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 130d1d6b-df21-5081-a28d-4d4adc17a5d6 with scrape type 1
2025-12-05 20:01:07 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 130d1d6b-df21-5081-a28d-4d4adc17a5d6
2025-12-05 20:01:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F243%252F28c3b44e3fb74d719bc92ff26fdee881%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:07 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/243/28c3b44e3fb74d719bc92ff26fdee881/images/large/1.jpg not downloaded yet
2025-12-05 20:01:07 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:07 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:07 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:07 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b051dc10>
2025-12-05 20:01:07 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:07 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:07 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:01:07 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:07 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:07 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:07 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_e0fd7b43-fbd3-5d03-a00c-671ccc573ab4/e0fd7b43-fbd3-5d03-a00c-671ccc573ab4_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F536%252F7ce0d532ce5940239ed7784fa7abb61a%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:07 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/536/7ce0d532ce5940239ed7784fa7abb61a/images/large/1.jpg saved to cache
2025-12-05 20:01:07 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item e0fd7b43-fbd3-5d03-a00c-671ccc573ab4
2025-12-05 20:01:07 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: e0fd7b43-fbd3-5d03-a00c-671ccc573ab4, skipping ID generation
2025-12-05 20:01:07 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e0fd7b43-fbd3-5d03-a00c-671ccc573ab4 with scrape type 1
2025-12-05 20:01:07 [crawlers.middlewares.photo_download_middleware] INFO: Removing item e0fd7b43-fbd3-5d03-a00c-671ccc573ab4 from tracking dictionaries
2025-12-05 20:01:07 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item e0fd7b43-fbd3-5d03-a00c-671ccc573ab4 sending to next pipeline
2025-12-05 20:01:07 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: e0fd7b43-fbd3-5d03-a00c-671ccc573ab4, identifier: 1788771
2025-12-05 20:01:07 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:07 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:07 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:07 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0468b10>
2025-12-05 20:01:07 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:07 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:08 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:08 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:08 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:08 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:08 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_562b2353-64f1-5e52-a46f-adb192edfff9/562b2353-64f1-5e52-a46f-adb192edfff9_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:08 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:08 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F760%252F2ebe9dd8d1104f18a9ea65f2f29739b5%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:08 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/760/2ebe9dd8d1104f18a9ea65f2f29739b5/images/large/1.jpg saved to cache
2025-12-05 20:01:08 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 562b2353-64f1-5e52-a46f-adb192edfff9
2025-12-05 20:01:08 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 562b2353-64f1-5e52-a46f-adb192edfff9, skipping ID generation
2025-12-05 20:01:08 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 562b2353-64f1-5e52-a46f-adb192edfff9 with scrape type 1
2025-12-05 20:01:08 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 562b2353-64f1-5e52-a46f-adb192edfff9 from tracking dictionaries
2025-12-05 20:01:08 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 562b2353-64f1-5e52-a46f-adb192edfff9 sending to next pipeline
2025-12-05 20:01:08 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 562b2353-64f1-5e52-a46f-adb192edfff9, identifier: 1788770
2025-12-05 20:01:08 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:08 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:08 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:08 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0451790>
2025-12-05 20:01:08 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:08 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:08 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:01:08 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:08 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:08 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:08 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_20fe41fd-d76a-565f-b590-2dd0feb845f0/20fe41fd-d76a-565f-b590-2dd0feb845f0_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:08 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:08 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F267%252Fbbe6679ac0ab4398a62617fe218b8203%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:08 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/267/bbe6679ac0ab4398a62617fe218b8203/images/large/1.jpg saved to cache
2025-12-05 20:01:08 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 20fe41fd-d76a-565f-b590-2dd0feb845f0
2025-12-05 20:01:08 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 20fe41fd-d76a-565f-b590-2dd0feb845f0, skipping ID generation
2025-12-05 20:01:08 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 20fe41fd-d76a-565f-b590-2dd0feb845f0 with scrape type 1
2025-12-05 20:01:08 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 20fe41fd-d76a-565f-b590-2dd0feb845f0 from tracking dictionaries
2025-12-05 20:01:08 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 20fe41fd-d76a-565f-b590-2dd0feb845f0 sending to next pipeline
2025-12-05 20:01:08 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 20fe41fd-d76a-565f-b590-2dd0feb845f0, identifier: 1788772
2025-12-05 20:01:08 [scrapy.extensions.logstats] INFO: Crawled 4 pages (at 4 pages/min), scraped 22 items (at 22 items/min)
2025-12-05 20:01:08 [ayvens.es] INFO: Found listing with ID: 1788780
2025-12-05 20:01:08 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:08 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788780%27 HTTP/1.1" 200 None
2025-12-05 20:01:08 [ayvens.es] INFO: Scrape type for 1788780: 1
2025-12-05 20:01:08 [ayvens.es] INFO: Extracting data for car ID: 1788780
2025-12-05 20:01:08 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: ea1a3343-8b33-5b00-84c4-85bfc920a4eb with identifier: 1788780
2025-12-05 20:01:08 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ea1a3343-8b33-5b00-84c4-85bfc920a4eb with scrape type 1
2025-12-05 20:01:08 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item ea1a3343-8b33-5b00-84c4-85bfc920a4eb
2025-12-05 20:01:08 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:08 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F247%252F0654d8a66fb249ebaf9df93bc71728da%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:08 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/247/0654d8a66fb249ebaf9df93bc71728da/images/large/1.jpg not downloaded yet
2025-12-05 20:01:08 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: e0fd7b43-fbd3-5d03-a00c-671ccc573ab4 with identifier: 1788771
2025-12-05 20:01:08 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: e0fd7b43-fbd3-5d03-a00c-671ccc573ab4 with identifier: 1788771
2025-12-05 20:01:08 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: e0fd7b43-fbd3-5d03-a00c-671ccc573ab4 with identifier: 1788771 to the API
2025-12-05 20:01:08 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:09 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:09 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/536/7ce0d532ce5940239ed7784fa7abb61a/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 140,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.9L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'e0fd7b43-fbd3-5d03-a00c-671ccc573ab4',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'MERCEDES-BENZ',
 'model': 'GLA',
 'odometer': 57666,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_e0fd7b43-fbd3-5d03-a00c-671ccc573ab4/e0fd7b43-fbd3-5d03-a00c-671ccc573ab4_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 285827,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/536/7ce0d532ce5940239ed7784fa7abb61a/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2022-07-28',
 'seats_number': 5,
 'seller_name': None,
 'title': 'MERCEDES-BENZ GLA GLA 200 D',
 'transmission': 'manual',
 'trim': 'GLA 200 D',
 'vin': None,
 'year': 2022}
2025-12-05 20:01:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788771%27 HTTP/1.1" 200 None
2025-12-05 20:01:09 [ayvens.es] INFO: Saving data for 1788771: {'created_time': 1764964869.19101, 'last_price_update_time': 1764964869.191025, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788771') HTTP/1.1" 204 0
2025-12-05 20:01:09 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 562b2353-64f1-5e52-a46f-adb192edfff9 with identifier: 1788770
2025-12-05 20:01:09 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 562b2353-64f1-5e52-a46f-adb192edfff9 with identifier: 1788770
2025-12-05 20:01:09 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'renault' containing 'renault' and model 'kangoo' containing 'kangoo'
2025-12-05 20:01:09 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 562b2353-64f1-5e52-a46f-adb192edfff9 spider: ayvens.es
2025-12-05 20:01:09 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 562b2353-64f1-5e52-a46f-adb192edfff9 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'renault' containing 'renault' and model 'kangoo' containing 'kangoo'
{'auction_title': '',
 'c_o_2_emission_value': 112,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '562b2353-64f1-5e52-a46f-adb192edfff9',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'KANGOO',
 'odometer': 113308,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_562b2353-64f1-5e52-a46f-adb192edfff9/562b2353-64f1-5e52-a46f-adb192edfff9_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 561058,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/760/2ebe9dd8d1104f18a9ea65f2f29739b5/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2018-05-09',
 'seats_number': 5,
 'seller_name': None,
 'title': 'RENAULT KANGOO Profesional M1-AF En. dCi 66kW (90CV) E6',
 'transmission': 'manual',
 'trim': 'Profesional M1-AF En. dCi 66kW (90CV) E6',
 'vin': None,
 'year': 2018}
2025-12-05 20:01:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788770%27 HTTP/1.1" 200 None
2025-12-05 20:01:09 [ayvens.es] INFO: Saving data for 1788770: {'created_time': 1764964869.603601, 'last_price_update_time': 1764964869.60361, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788770') HTTP/1.1" 204 0
2025-12-05 20:01:09 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 20fe41fd-d76a-565f-b590-2dd0feb845f0 with identifier: 1788772
2025-12-05 20:01:09 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 20fe41fd-d76a-565f-b590-2dd0feb845f0 with identifier: 1788772
2025-12-05 20:01:09 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 20fe41fd-d76a-565f-b590-2dd0feb845f0 with identifier: 1788772 to the API
2025-12-05 20:01:09 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:09 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:09 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/267/bbe6679ac0ab4398a62617fe218b8203/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 113,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '20fe41fd-d76a-565f-b590-2dd0feb845f0',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SEAT',
 'model': 'LEON',
 'odometer': 94909,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_20fe41fd-d76a-565f-b590-2dd0feb845f0/20fe41fd-d76a-565f-b590-2dd0feb845f0_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 348763,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/267/bbe6679ac0ab4398a62617fe218b8203/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-10-08',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SEAT LEON 2.0 TDI 85kW S&S Style Go',
 'transmission': 'manual',
 'trim': '2.0 TDI 85kW S&S Style Go',
 'vin': None,
 'year': 2021}
2025-12-05 20:01:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788772%27 HTTP/1.1" 200 None
2025-12-05 20:01:09 [ayvens.es] INFO: Saving data for 1788772: {'created_time': 1764964869.992806, 'last_price_update_time': 1764964869.992818, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788772') HTTP/1.1" 204 0
2025-12-05 20:01:10 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/574/f91e81c4443b460eb54886f5fd6be60d/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:10 [ayvens.es] INFO: Found listing with ID: 1788781
2025-12-05 20:01:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788781%27 HTTP/1.1" 200 None
2025-12-05 20:01:10 [ayvens.es] INFO: Scrape type for 1788781: 1
2025-12-05 20:01:10 [ayvens.es] INFO: Extracting data for car ID: 1788781
2025-12-05 20:01:10 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: ac293a8d-cce9-5358-b34d-995800cc833a with identifier: 1788781
2025-12-05 20:01:10 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ac293a8d-cce9-5358-b34d-995800cc833a with scrape type 1
2025-12-05 20:01:10 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item ac293a8d-cce9-5358-b34d-995800cc833a
2025-12-05 20:01:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F570%252F4d46cd0019ee483ba0ae6faa5a85663e%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:10 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/570/4d46cd0019ee483ba0ae6faa5a85663e/images/large/1.jpg not downloaded yet
2025-12-05 20:01:10 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:10 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:10 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:10 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0acf0d0>
2025-12-05 20:01:10 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:10 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:10 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1969
2025-12-05 20:01:10 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:10 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:10 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:10 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_5e68e2ec-0c20-565b-b28e-711cc82caee0/5e68e2ec-0c20-565b-b28e-711cc82caee0_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F193%252F721deb3d172045179808e8c1b1dd00a9%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:10 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/193/721deb3d172045179808e8c1b1dd00a9/images/large/1.jpg saved to cache
2025-12-05 20:01:10 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 5e68e2ec-0c20-565b-b28e-711cc82caee0
2025-12-05 20:01:10 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 5e68e2ec-0c20-565b-b28e-711cc82caee0, skipping ID generation
2025-12-05 20:01:10 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 5e68e2ec-0c20-565b-b28e-711cc82caee0 with scrape type 1
2025-12-05 20:01:10 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 5e68e2ec-0c20-565b-b28e-711cc82caee0 from tracking dictionaries
2025-12-05 20:01:10 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 5e68e2ec-0c20-565b-b28e-711cc82caee0 sending to next pipeline
2025-12-05 20:01:10 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 5e68e2ec-0c20-565b-b28e-711cc82caee0, identifier: 1788773
2025-12-05 20:01:10 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:10 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:10 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:10 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0d5ec10>
2025-12-05 20:01:10 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:10 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:10 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:10 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:10 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:10 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:10 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_cc08d996-1736-599d-9047-d85adec3cc5a/cc08d996-1736-599d-9047-d85adec3cc5a_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F870%252F40d22532ee614c4e9cc068484630ee00%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:10 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/870/40d22532ee614c4e9cc068484630ee00/images/large/1.jpg saved to cache
2025-12-05 20:01:10 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item cc08d996-1736-599d-9047-d85adec3cc5a
2025-12-05 20:01:10 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: cc08d996-1736-599d-9047-d85adec3cc5a, skipping ID generation
2025-12-05 20:01:10 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item cc08d996-1736-599d-9047-d85adec3cc5a with scrape type 1
2025-12-05 20:01:10 [crawlers.middlewares.photo_download_middleware] INFO: Removing item cc08d996-1736-599d-9047-d85adec3cc5a from tracking dictionaries
2025-12-05 20:01:10 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item cc08d996-1736-599d-9047-d85adec3cc5a sending to next pipeline
2025-12-05 20:01:10 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: cc08d996-1736-599d-9047-d85adec3cc5a, identifier: 1788774
2025-12-05 20:01:10 [ayvens.es] INFO: Found listing with ID: 1788782
2025-12-05 20:01:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788782%27 HTTP/1.1" 200 None
2025-12-05 20:01:10 [ayvens.es] INFO: Scrape type for 1788782: 1
2025-12-05 20:01:10 [ayvens.es] INFO: Extracting data for car ID: 1788782
2025-12-05 20:01:10 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: e66bf8d3-822f-5e7e-97cf-38eff6b594ea with identifier: 1788782
2025-12-05 20:01:10 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e66bf8d3-822f-5e7e-97cf-38eff6b594ea with scrape type 1
2025-12-05 20:01:10 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item e66bf8d3-822f-5e7e-97cf-38eff6b594ea
2025-12-05 20:01:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F797%252Fb4dad849b8934076a422bf7164776e96%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:11 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/797/b4dad849b8934076a422bf7164776e96/images/large/1.jpg not downloaded yet
2025-12-05 20:01:11 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 5e68e2ec-0c20-565b-b28e-711cc82caee0 with identifier: 1788773
2025-12-05 20:01:11 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 5e68e2ec-0c20-565b-b28e-711cc82caee0 with identifier: 1788773
2025-12-05 20:01:11 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 5e68e2ec-0c20-565b-b28e-711cc82caee0 with identifier: 1788773 to the API
2025-12-05 20:01:11 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:11 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:11 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/193/721deb3d172045179808e8c1b1dd00a9/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 130,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.6L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '5e68e2ec-0c20-565b-b28e-711cc82caee0',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'KIA',
 'model': 'SPORTAGE',
 'odometer': 66539,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_5e68e2ec-0c20-565b-b28e-711cc82caee0/5e68e2ec-0c20-565b-b28e-711cc82caee0_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 343288,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/193/721deb3d172045179808e8c1b1dd00a9/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2018-12-29',
 'seats_number': 5,
 'seller_name': None,
 'title': 'KIA SPORTAGE 1.6 CRDi VGT 85kW (115CV) Business 4x2',
 'transmission': 'manual',
 'trim': '1.6 CRDi VGT 85kW (115CV) Business 4x2',
 'vin': None,
 'year': 2018}
2025-12-05 20:01:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788773%27 HTTP/1.1" 200 None
2025-12-05 20:01:11 [ayvens.es] INFO: Saving data for 1788773: {'created_time': 1764964871.33905, 'last_price_update_time': 1764964871.339062, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788773') HTTP/1.1" 204 0
2025-12-05 20:01:11 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: cc08d996-1736-599d-9047-d85adec3cc5a with identifier: 1788774
2025-12-05 20:01:11 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: cc08d996-1736-599d-9047-d85adec3cc5a with identifier: 1788774
2025-12-05 20:01:11 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: cc08d996-1736-599d-9047-d85adec3cc5a with identifier: 1788774 to the API
2025-12-05 20:01:11 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:11 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:11 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/870/40d22532ee614c4e9cc068484630ee00/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 113,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'cc08d996-1736-599d-9047-d85adec3cc5a',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SEAT',
 'model': 'LEON',
 'odometer': 92784,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_cc08d996-1736-599d-9047-d85adec3cc5a/cc08d996-1736-599d-9047-d85adec3cc5a_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 325454,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/870/40d22532ee614c4e9cc068484630ee00/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-10-15',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SEAT LEON 2.0 TDI 85kW S&S Style Go',
 'transmission': 'manual',
 'trim': '2.0 TDI 85kW S&S Style Go',
 'vin': None,
 'year': 2021}
2025-12-05 20:01:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788774%27 HTTP/1.1" 200 None
2025-12-05 20:01:11 [ayvens.es] INFO: Saving data for 1788774: {'created_time': 1764964871.707831, 'last_price_update_time': 1764964871.707842, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788774') HTTP/1.1" 204 0
2025-12-05 20:01:11 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/972/294a756b02b54c7089ad2664405d7098/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:11 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/525/24b2fea0f5b840d2abb5c43b6e60bfba/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:11 [ayvens.es] INFO: Found listing with ID: 1788783
2025-12-05 20:01:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788783%27 HTTP/1.1" 200 None
2025-12-05 20:01:11 [ayvens.es] INFO: Scrape type for 1788783: 1
2025-12-05 20:01:11 [ayvens.es] INFO: Extracting data for car ID: 1788783
2025-12-05 20:01:11 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 950afb39-23df-5e09-81d2-479bc21e91db with identifier: 1788783
2025-12-05 20:01:11 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 950afb39-23df-5e09-81d2-479bc21e91db with scrape type 1
2025-12-05 20:01:11 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 950afb39-23df-5e09-81d2-479bc21e91db
2025-12-05 20:01:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F232%252F503abfd227df4755bd0662aed6350753%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:11 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/232/503abfd227df4755bd0662aed6350753/images/large/1.jpg not downloaded yet
2025-12-05 20:01:11 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:11 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:11 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:11 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b05199d0>
2025-12-05 20:01:11 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:11 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:11 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1969
2025-12-05 20:01:11 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:12 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:12 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:12 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_374218cf-74ac-5703-9e62-bb769f58be63/374218cf-74ac-5703-9e62-bb769f58be63_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F574%252Ff91e81c4443b460eb54886f5fd6be60d%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:12 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/574/f91e81c4443b460eb54886f5fd6be60d/images/large/1.jpg saved to cache
2025-12-05 20:01:12 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 374218cf-74ac-5703-9e62-bb769f58be63
2025-12-05 20:01:12 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 374218cf-74ac-5703-9e62-bb769f58be63, skipping ID generation
2025-12-05 20:01:12 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 374218cf-74ac-5703-9e62-bb769f58be63 with scrape type 1
2025-12-05 20:01:12 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 374218cf-74ac-5703-9e62-bb769f58be63 from tracking dictionaries
2025-12-05 20:01:12 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 374218cf-74ac-5703-9e62-bb769f58be63 sending to next pipeline
2025-12-05 20:01:12 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 374218cf-74ac-5703-9e62-bb769f58be63, identifier: 1788775
2025-12-05 20:01:12 [ayvens.es] INFO: Found listing with ID: 1788784
2025-12-05 20:01:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788784%27 HTTP/1.1" 200 None
2025-12-05 20:01:12 [ayvens.es] INFO: Scrape type for 1788784: 1
2025-12-05 20:01:12 [ayvens.es] INFO: Extracting data for car ID: 1788784
2025-12-05 20:01:12 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: db150b0c-38e8-5e79-a618-d63d29fcc23b with identifier: 1788784
2025-12-05 20:01:12 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item db150b0c-38e8-5e79-a618-d63d29fcc23b with scrape type 1
2025-12-05 20:01:12 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item db150b0c-38e8-5e79-a618-d63d29fcc23b
2025-12-05 20:01:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F809%252F91ff773b16544485902f3995d64b59fa%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:12 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/809/91ff773b16544485902f3995d64b59fa/images/large/1.jpg not downloaded yet
2025-12-05 20:01:12 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 374218cf-74ac-5703-9e62-bb769f58be63 with identifier: 1788775
2025-12-05 20:01:12 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 374218cf-74ac-5703-9e62-bb769f58be63 with identifier: 1788775
2025-12-05 20:01:12 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: 143263 is greater than the maximum of 130000

Failed validating 'maximum' in schema['allOf'][0]['then']['properties']['odometer_in_km']:
    {'maximum': 130000, 'type': 'integer'}

On instance['odometer_in_km']:
    143263
2025-12-05 20:01:12 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE_FOR_MAKE failed for item: 374218cf-74ac-5703-9e62-bb769f58be63 spider: ayvens.es
2025-12-05 20:01:12 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 374218cf-74ac-5703-9e62-bb769f58be63 due to rule: high_mileage_for_make.json, error: 143263 is greater than the maximum of 130000

Failed validating 'maximum' in schema['allOf'][0]['then']['properties']['odometer_in_km']:
    {'maximum': 130000, 'type': 'integer'}

On instance['odometer_in_km']:
    143263
{'auction_title': '',
 'c_o_2_emission_value': 104,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '374218cf-74ac-5703-9e62-bb769f58be63',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'CLIO',
 'odometer': 143263,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_374218cf-74ac-5703-9e62-bb769f58be63/374218cf-74ac-5703-9e62-bb769f58be63_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 357335,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/574/f91e81c4443b460eb54886f5fd6be60d/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2018-07-03',
 'seats_number': 5,
 'seller_name': None,
 'title': 'RENAULT CLIO Business Energy dCi 55kW (75CV) -18',
 'transmission': 'manual',
 'trim': 'Business Energy dCi 55kW (75CV) -18',
 'vin': None,
 'year': 2018}
2025-12-05 20:01:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788775%27 HTTP/1.1" 200 None
2025-12-05 20:01:12 [ayvens.es] INFO: Saving data for 1788775: {'created_time': 1764964872.541213, 'last_price_update_time': 1764964872.541221, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788775') HTTP/1.1" 204 0
2025-12-05 20:01:12 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/243/28c3b44e3fb74d719bc92ff26fdee881/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:12 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/397/d576a5b8b43546fbae5ba93d4f8cf27c/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:12 [ayvens.es] INFO: Found listing with ID: 1788785
2025-12-05 20:01:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788785%27 HTTP/1.1" 200 None
2025-12-05 20:01:12 [ayvens.es] INFO: Scrape type for 1788785: 1
2025-12-05 20:01:12 [ayvens.es] INFO: Extracting data for car ID: 1788785
2025-12-05 20:01:12 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 6894e7db-d32e-5705-bee1-ad3ca9ee3b32 with identifier: 1788785
2025-12-05 20:01:12 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6894e7db-d32e-5705-bee1-ad3ca9ee3b32 with scrape type 1
2025-12-05 20:01:12 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 6894e7db-d32e-5705-bee1-ad3ca9ee3b32
2025-12-05 20:01:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F331%252F12ca9c587e2e4987bdbad681e08c30c5%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:12 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/331/12ca9c587e2e4987bdbad681e08c30c5/images/large/1.jpg not downloaded yet
2025-12-05 20:01:12 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:12 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:12 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:12 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0459dd0>
2025-12-05 20:01:12 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:12 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:12 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:01:12 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:12 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:12 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:12 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_f6efab77-d6e4-549d-8cf8-ecc622dde6fe/f6efab77-d6e4-549d-8cf8-ecc622dde6fe_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F972%252F294a756b02b54c7089ad2664405d7098%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:12 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/972/294a756b02b54c7089ad2664405d7098/images/large/1.jpg saved to cache
2025-12-05 20:01:12 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item f6efab77-d6e4-549d-8cf8-ecc622dde6fe
2025-12-05 20:01:12 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: f6efab77-d6e4-549d-8cf8-ecc622dde6fe, skipping ID generation
2025-12-05 20:01:12 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f6efab77-d6e4-549d-8cf8-ecc622dde6fe with scrape type 1
2025-12-05 20:01:12 [crawlers.middlewares.photo_download_middleware] INFO: Removing item f6efab77-d6e4-549d-8cf8-ecc622dde6fe from tracking dictionaries
2025-12-05 20:01:12 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item f6efab77-d6e4-549d-8cf8-ecc622dde6fe sending to next pipeline
2025-12-05 20:01:13 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: f6efab77-d6e4-549d-8cf8-ecc622dde6fe, identifier: 1788777
2025-12-05 20:01:13 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:13 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:13 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:13 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b054c650>
2025-12-05 20:01:13 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:13 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:13 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1971
2025-12-05 20:01:13 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:13 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:13 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:13 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_06434433-7dd4-50df-9d22-108f90d4c411/06434433-7dd4-50df-9d22-108f90d4c411_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F525%252F24b2fea0f5b840d2abb5c43b6e60bfba%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:13 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/525/24b2fea0f5b840d2abb5c43b6e60bfba/images/large/1.jpg saved to cache
2025-12-05 20:01:13 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 06434433-7dd4-50df-9d22-108f90d4c411
2025-12-05 20:01:13 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 06434433-7dd4-50df-9d22-108f90d4c411, skipping ID generation
2025-12-05 20:01:13 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 06434433-7dd4-50df-9d22-108f90d4c411 with scrape type 1
2025-12-05 20:01:13 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 06434433-7dd4-50df-9d22-108f90d4c411 from tracking dictionaries
2025-12-05 20:01:13 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 06434433-7dd4-50df-9d22-108f90d4c411 sending to next pipeline
2025-12-05 20:01:13 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 06434433-7dd4-50df-9d22-108f90d4c411, identifier: 1788776
2025-12-05 20:01:13 [ayvens.es] INFO: Found listing with ID: 1788786
2025-12-05 20:01:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788786%27 HTTP/1.1" 200 None
2025-12-05 20:01:13 [ayvens.es] INFO: Scrape type for 1788786: 1
2025-12-05 20:01:13 [ayvens.es] INFO: Extracting data for car ID: 1788786
2025-12-05 20:01:13 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 5ff4b563-d73a-5f7a-85d7-97c0d5c7dbdb with identifier: 1788786
2025-12-05 20:01:13 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 5ff4b563-d73a-5f7a-85d7-97c0d5c7dbdb with scrape type 1
2025-12-05 20:01:13 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 5ff4b563-d73a-5f7a-85d7-97c0d5c7dbdb
2025-12-05 20:01:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F086%252F70cb875d589d4d87a16bca97f36ed74d%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:13 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/086/70cb875d589d4d87a16bca97f36ed74d/images/large/1.jpg not downloaded yet
2025-12-05 20:01:13 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: f6efab77-d6e4-549d-8cf8-ecc622dde6fe with identifier: 1788777
2025-12-05 20:01:13 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: f6efab77-d6e4-549d-8cf8-ecc622dde6fe with identifier: 1788777
2025-12-05 20:01:13 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: f6efab77-d6e4-549d-8cf8-ecc622dde6fe with identifier: 1788777 to the API
2025-12-05 20:01:13 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:14 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:14 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/972/294a756b02b54c7089ad2664405d7098/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 125,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'f6efab77-d6e4-549d-8cf8-ecc622dde6fe',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'CITROËN',
 'model': 'C3 AIRCROSS',
 'odometer': 60353,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_f6efab77-d6e4-549d-8cf8-ecc622dde6fe/f6efab77-d6e4-549d-8cf8-ecc622dde6fe_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 317682,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/972/294a756b02b54c7089ad2664405d7098/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2022-12-27',
 'seats_number': 5,
 'seller_name': None,
 'title': 'CITROËN C3 AIRCROSS BlueHDi 81kW (110CV) S&S Feel Pack',
 'transmission': 'manual',
 'trim': 'BlueHDi 81kW (110CV) S&S Feel Pack',
 'vin': None,
 'year': 2022}
2025-12-05 20:01:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788777%27 HTTP/1.1" 200 None
2025-12-05 20:01:14 [ayvens.es] INFO: Saving data for 1788777: {'created_time': 1764964874.522034, 'last_price_update_time': 1764964874.522047, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788777') HTTP/1.1" 204 0
2025-12-05 20:01:14 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 06434433-7dd4-50df-9d22-108f90d4c411 with identifier: 1788776
2025-12-05 20:01:14 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 06434433-7dd4-50df-9d22-108f90d4c411 with identifier: 1788776
2025-12-05 20:01:14 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 06434433-7dd4-50df-9d22-108f90d4c411 with identifier: 1788776 to the API
2025-12-05 20:01:14 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:14 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:14 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/525/24b2fea0f5b840d2abb5c43b6e60bfba/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 113,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '06434433-7dd4-50df-9d22-108f90d4c411',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SEAT',
 'model': 'LEON',
 'odometer': 149757,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_06434433-7dd4-50df-9d22-108f90d4c411/06434433-7dd4-50df-9d22-108f90d4c411_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 349188,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/525/24b2fea0f5b840d2abb5c43b6e60bfba/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-07-06',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SEAT LEON 2.0 TDI 85kW S&S Style Go',
 'transmission': 'manual',
 'trim': '2.0 TDI 85kW S&S Style Go',
 'vin': None,
 'year': 2021}
2025-12-05 20:01:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788776%27 HTTP/1.1" 200 None
2025-12-05 20:01:14 [ayvens.es] INFO: Saving data for 1788776: {'created_time': 1764964874.927197, 'last_price_update_time': 1764964874.927209, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788776') HTTP/1.1" 204 0
2025-12-05 20:01:14 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/570/4d46cd0019ee483ba0ae6faa5a85663e/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:14 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/247/0654d8a66fb249ebaf9df93bc71728da/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:14 [ayvens.es] INFO: Found listing with ID: 1788787
2025-12-05 20:01:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788787%27 HTTP/1.1" 200 None
2025-12-05 20:01:14 [ayvens.es] INFO: Scrape type for 1788787: 1
2025-12-05 20:01:14 [ayvens.es] INFO: Extracting data for car ID: 1788787
2025-12-05 20:01:14 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: c290bec7-2c63-5f76-9426-c6705be56ab0 with identifier: 1788787
2025-12-05 20:01:14 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item c290bec7-2c63-5f76-9426-c6705be56ab0 with scrape type 1
2025-12-05 20:01:14 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item c290bec7-2c63-5f76-9426-c6705be56ab0
2025-12-05 20:01:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F738%252F8a930f8abd15444e8e4e28a61ca938ad%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:14 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/738/8a930f8abd15444e8e4e28a61ca938ad/images/large/1.jpg not downloaded yet
2025-12-05 20:01:14 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:14 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:14 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:14 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b04b6690>
2025-12-05 20:01:14 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:15 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:15 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:01:15 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:15 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:15 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:15 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_130d1d6b-df21-5081-a28d-4d4adc17a5d6/130d1d6b-df21-5081-a28d-4d4adc17a5d6_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F243%252F28c3b44e3fb74d719bc92ff26fdee881%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:15 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/243/28c3b44e3fb74d719bc92ff26fdee881/images/large/1.jpg saved to cache
2025-12-05 20:01:15 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 130d1d6b-df21-5081-a28d-4d4adc17a5d6
2025-12-05 20:01:15 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 130d1d6b-df21-5081-a28d-4d4adc17a5d6, skipping ID generation
2025-12-05 20:01:15 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 130d1d6b-df21-5081-a28d-4d4adc17a5d6 with scrape type 1
2025-12-05 20:01:15 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 130d1d6b-df21-5081-a28d-4d4adc17a5d6 from tracking dictionaries
2025-12-05 20:01:15 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 130d1d6b-df21-5081-a28d-4d4adc17a5d6 sending to next pipeline
2025-12-05 20:01:15 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 130d1d6b-df21-5081-a28d-4d4adc17a5d6, identifier: 1788779
2025-12-05 20:01:15 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:15 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:15 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:15 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b53c0490>
2025-12-05 20:01:15 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:15 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:15 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:15 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:15 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:15 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:15 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_9c269e51-8b6e-56ed-8498-36d194208f53/9c269e51-8b6e-56ed-8498-36d194208f53_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F397%252Fd576a5b8b43546fbae5ba93d4f8cf27c%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:15 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/397/d576a5b8b43546fbae5ba93d4f8cf27c/images/large/1.jpg saved to cache
2025-12-05 20:01:15 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 9c269e51-8b6e-56ed-8498-36d194208f53
2025-12-05 20:01:15 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 9c269e51-8b6e-56ed-8498-36d194208f53, skipping ID generation
2025-12-05 20:01:15 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9c269e51-8b6e-56ed-8498-36d194208f53 with scrape type 1
2025-12-05 20:01:15 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 9c269e51-8b6e-56ed-8498-36d194208f53 from tracking dictionaries
2025-12-05 20:01:15 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 9c269e51-8b6e-56ed-8498-36d194208f53 sending to next pipeline
2025-12-05 20:01:16 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 9c269e51-8b6e-56ed-8498-36d194208f53, identifier: 1788778
2025-12-05 20:01:16 [ayvens.es] INFO: Found listing with ID: 1788788
2025-12-05 20:01:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788788%27 HTTP/1.1" 200 None
2025-12-05 20:01:16 [ayvens.es] INFO: Scrape type for 1788788: 1
2025-12-05 20:01:16 [ayvens.es] INFO: Extracting data for car ID: 1788788
2025-12-05 20:01:16 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: fe518f07-ed7e-5596-a2bd-38018f32eadf with identifier: 1788788
2025-12-05 20:01:16 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item fe518f07-ed7e-5596-a2bd-38018f32eadf with scrape type 1
2025-12-05 20:01:16 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item fe518f07-ed7e-5596-a2bd-38018f32eadf
2025-12-05 20:01:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F604%252Fba1fa4a70626424a88178d1b65a3ed97%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:16 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/604/ba1fa4a70626424a88178d1b65a3ed97/images/large/1.jpg not downloaded yet
2025-12-05 20:01:16 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 130d1d6b-df21-5081-a28d-4d4adc17a5d6 with identifier: 1788779
2025-12-05 20:01:16 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:01:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:01:16 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 130d1d6b-df21-5081-a28d-4d4adc17a5d6 with identifier: 1788779
2025-12-05 20:01:16 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: 185917 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    185917
2025-12-05 20:01:16 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE_FOR_COUNTRY failed for item: 130d1d6b-df21-5081-a28d-4d4adc17a5d6 spider: ayvens.es
2025-12-05 20:01:16 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 130d1d6b-df21-5081-a28d-4d4adc17a5d6 due to rule: high_mileage_for_country.json, error: 185917 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    185917
{'auction_title': '',
 'c_o_2_emission_value': 134,
 'color': 'Mineralgrau',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '130d1d6b-df21-5081-a28d-4d4adc17a5d6',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'BMW',
 'model': 'X1',
 'odometer': 185917,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_130d1d6b-df21-5081-a28d-4d4adc17a5d6/130d1d6b-df21-5081-a28d-4d4adc17a5d6_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 349562,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/243/28c3b44e3fb74d719bc92ff26fdee881/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-10-26',
 'seats_number': 5,
 'seller_name': None,
 'title': 'BMW X1 sDrive16d',
 'transmission': 'manual',
 'trim': 'sDrive16d',
 'vin': None,
 'year': 2021}
2025-12-05 20:01:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788779%27 HTTP/1.1" 200 None
2025-12-05 20:01:16 [ayvens.es] INFO: Saving data for 1788779: {'created_time': 1764964876.369988, 'last_price_update_time': 1764964876.369998, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788779') HTTP/1.1" 204 0
2025-12-05 20:01:16 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 9c269e51-8b6e-56ed-8498-36d194208f53 with identifier: 1788778
2025-12-05 20:01:16 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 9c269e51-8b6e-56ed-8498-36d194208f53 with identifier: 1788778
2025-12-05 20:01:16 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 9c269e51-8b6e-56ed-8498-36d194208f53 with identifier: 1788778 to the API
2025-12-05 20:01:16 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:16 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:16 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/397/d576a5b8b43546fbae5ba93d4f8cf27c/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 117,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '9c269e51-8b6e-56ed-8498-36d194208f53',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'CLIO',
 'odometer': 58444,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_9c269e51-8b6e-56ed-8498-36d194208f53/9c269e51-8b6e-56ed-8498-36d194208f53_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 347462,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/397/d576a5b8b43546fbae5ba93d4f8cf27c/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-10-14',
 'seats_number': 5,
 'seller_name': None,
 'title': 'RENAULT CLIO Business SCe 49 kW (67CV)',
 'transmission': 'manual',
 'trim': 'Business SCe 49 kW (67CV)',
 'vin': None,
 'year': 2021}
2025-12-05 20:01:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788778%27 HTTP/1.1" 200 None
2025-12-05 20:01:16 [ayvens.es] INFO: Saving data for 1788778: {'created_time': 1764964876.70477, 'last_price_update_time': 1764964876.704784, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788778') HTTP/1.1" 204 0
2025-12-05 20:01:16 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/232/503abfd227df4755bd0662aed6350753/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:16 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/797/b4dad849b8934076a422bf7164776e96/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:16 [ayvens.es] INFO: Found listing with ID: 1788789
2025-12-05 20:01:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788789%27 HTTP/1.1" 200 None
2025-12-05 20:01:16 [ayvens.es] INFO: Scrape type for 1788789: 1
2025-12-05 20:01:16 [ayvens.es] INFO: Extracting data for car ID: 1788789
2025-12-05 20:01:16 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 5b91fcd0-6c3f-5c77-b7ff-68d5654d467d with identifier: 1788789
2025-12-05 20:01:16 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 5b91fcd0-6c3f-5c77-b7ff-68d5654d467d with scrape type 1
2025-12-05 20:01:16 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 5b91fcd0-6c3f-5c77-b7ff-68d5654d467d
2025-12-05 20:01:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F751%252F886bd7e5a8c948448d173199c52b1f40%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:16 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/751/886bd7e5a8c948448d173199c52b1f40/images/large/1.jpg not downloaded yet
2025-12-05 20:01:16 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:16 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:16 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:16 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0441f50>
2025-12-05 20:01:16 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:16 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:16 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1969
2025-12-05 20:01:16 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:17 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_ac293a8d-cce9-5358-b34d-995800cc833a/ac293a8d-cce9-5358-b34d-995800cc833a_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F570%252F4d46cd0019ee483ba0ae6faa5a85663e%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:17 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/570/4d46cd0019ee483ba0ae6faa5a85663e/images/large/1.jpg saved to cache
2025-12-05 20:01:17 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item ac293a8d-cce9-5358-b34d-995800cc833a
2025-12-05 20:01:17 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: ac293a8d-cce9-5358-b34d-995800cc833a, skipping ID generation
2025-12-05 20:01:17 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ac293a8d-cce9-5358-b34d-995800cc833a with scrape type 1
2025-12-05 20:01:17 [crawlers.middlewares.photo_download_middleware] INFO: Removing item ac293a8d-cce9-5358-b34d-995800cc833a from tracking dictionaries
2025-12-05 20:01:17 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item ac293a8d-cce9-5358-b34d-995800cc833a sending to next pipeline
2025-12-05 20:01:17 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: ac293a8d-cce9-5358-b34d-995800cc833a, identifier: 1788781
2025-12-05 20:01:17 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:17 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:17 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:17 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b051fd50>
2025-12-05 20:01:17 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:17 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:17 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1969
2025-12-05 20:01:17 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:17 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_ea1a3343-8b33-5b00-84c4-85bfc920a4eb/ea1a3343-8b33-5b00-84c4-85bfc920a4eb_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F247%252F0654d8a66fb249ebaf9df93bc71728da%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:17 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/247/0654d8a66fb249ebaf9df93bc71728da/images/large/1.jpg saved to cache
2025-12-05 20:01:17 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item ea1a3343-8b33-5b00-84c4-85bfc920a4eb
2025-12-05 20:01:17 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: ea1a3343-8b33-5b00-84c4-85bfc920a4eb, skipping ID generation
2025-12-05 20:01:17 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ea1a3343-8b33-5b00-84c4-85bfc920a4eb with scrape type 1
2025-12-05 20:01:17 [crawlers.middlewares.photo_download_middleware] INFO: Removing item ea1a3343-8b33-5b00-84c4-85bfc920a4eb from tracking dictionaries
2025-12-05 20:01:17 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item ea1a3343-8b33-5b00-84c4-85bfc920a4eb sending to next pipeline
2025-12-05 20:01:17 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: ea1a3343-8b33-5b00-84c4-85bfc920a4eb, identifier: 1788780
2025-12-05 20:01:17 [ayvens.es] INFO: Found listing with ID: 1788790
2025-12-05 20:01:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788790%27 HTTP/1.1" 200 None
2025-12-05 20:01:17 [ayvens.es] INFO: Scrape type for 1788790: 1
2025-12-05 20:01:17 [ayvens.es] INFO: Extracting data for car ID: 1788790
2025-12-05 20:01:17 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 19941e70-7776-5c54-9017-3f86e20a4ae4 with identifier: 1788790
2025-12-05 20:01:17 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 19941e70-7776-5c54-9017-3f86e20a4ae4 with scrape type 1
2025-12-05 20:01:17 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 19941e70-7776-5c54-9017-3f86e20a4ae4
2025-12-05 20:01:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F080%252F67166f8b49d04340a54226538ee6ecf3%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:17 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/080/67166f8b49d04340a54226538ee6ecf3/images/large/1.jpg not downloaded yet
2025-12-05 20:01:17 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: ac293a8d-cce9-5358-b34d-995800cc833a with identifier: 1788781
2025-12-05 20:01:17 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: ac293a8d-cce9-5358-b34d-995800cc833a with identifier: 1788781
2025-12-05 20:01:17 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'negro', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'audi', 'model': 'q5', 'trim': 's line 35 tdi 120kw quattro s tronic', 'title': 'audi q5 s line 35 tdi 120kw quattro s tronic', 'price': 0, 'odometer': 232271, 'odometer_reading_unit': 'km', 'registration_date': '2020-08-26', 'year': 2020, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '2.0l', 'c_o_2_emission_value': 145, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'ac293a8d-cce9-5358-b34d-995800cc833a', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ac293a8d-cce9-5358-b34d-995800cc833a/ac293a8d-cce9-5358-b34d-995800cc833a_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/570/4d46cd0019ee483ba0ae6faa5a85663e/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 356650}], 'current_year_diff': 5, 'age_in_months': 63, 'make_model': 'audi q5', 'sanitized_make': 'audi', 'odometer_in_km': 232271}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 63,
     'auction_title': '',
     'c_o_2_emission_value': 145,
     'color': 'negro',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '2.0l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': 'ac293a8d-cce9-5358-b34d-995800cc833a',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'audi',
     'make_model': 'audi q5',
     'model': 'q5',
     'odometer': 232271,
     'odometer_in_km': 232271,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ac293a8d-cce9-5358-b34d-995800cc833a/ac293a8d-cce9-5358-b34d-995800cc833a_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 356650,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/570/4d46cd0019ee483ba0ae6faa5a85663e/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-08-26',
     'sanitized_make': 'audi',
     'seats_number': 5,
     'seller_name': None,
     'title': 'audi q5 s line 35 tdi 120kw quattro s tronic',
     'transmission': 'manual',
     'trim': 's line 35 tdi 120kw quattro s tronic',
     'vin': None,
     'year': 2020}
2025-12-05 20:01:17 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE failed for item: ac293a8d-cce9-5358-b34d-995800cc833a spider: ayvens.es
2025-12-05 20:01:18 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: ac293a8d-cce9-5358-b34d-995800cc833a due to rule: high_mileage.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'negro', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'audi', 'model': 'q5', 'trim': 's line 35 tdi 120kw quattro s tronic', 'title': 'audi q5 s line 35 tdi 120kw quattro s tronic', 'price': 0, 'odometer': 232271, 'odometer_reading_unit': 'km', 'registration_date': '2020-08-26', 'year': 2020, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '2.0l', 'c_o_2_emission_value': 145, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'ac293a8d-cce9-5358-b34d-995800cc833a', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ac293a8d-cce9-5358-b34d-995800cc833a/ac293a8d-cce9-5358-b34d-995800cc833a_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/570/4d46cd0019ee483ba0ae6faa5a85663e/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 356650}], 'current_year_diff': 5, 'age_in_months': 63, 'make_model': 'audi q5', 'sanitized_make': 'audi', 'odometer_in_km': 232271}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 63,
     'auction_title': '',
     'c_o_2_emission_value': 145,
     'color': 'negro',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '2.0l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': 'ac293a8d-cce9-5358-b34d-995800cc833a',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'audi',
     'make_model': 'audi q5',
     'model': 'q5',
     'odometer': 232271,
     'odometer_in_km': 232271,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ac293a8d-cce9-5358-b34d-995800cc833a/ac293a8d-cce9-5358-b34d-995800cc833a_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 356650,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/570/4d46cd0019ee483ba0ae6faa5a85663e/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-08-26',
     'sanitized_make': 'audi',
     'seats_number': 5,
     'seller_name': None,
     'title': 'audi q5 s line 35 tdi 120kw quattro s tronic',
     'transmission': 'manual',
     'trim': 's line 35 tdi 120kw quattro s tronic',
     'vin': None,
     'year': 2020}
{'auction_title': '',
 'c_o_2_emission_value': 145,
 'color': 'Negro',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'ac293a8d-cce9-5358-b34d-995800cc833a',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'AUDI',
 'model': 'Q5',
 'odometer': 232271,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ac293a8d-cce9-5358-b34d-995800cc833a/ac293a8d-cce9-5358-b34d-995800cc833a_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 356650,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/570/4d46cd0019ee483ba0ae6faa5a85663e/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-08-26',
 'seats_number': 5,
 'seller_name': None,
 'title': 'AUDI Q5 S line 35 TDI 120kW quattro S tronic',
 'transmission': 'manual',
 'trim': 'S line 35 TDI 120kW quattro S tronic',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788781%27 HTTP/1.1" 200 None
2025-12-05 20:01:18 [ayvens.es] INFO: Saving data for 1788781: {'created_time': 1764964878.05842, 'last_price_update_time': 1764964878.05843, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788781') HTTP/1.1" 204 0
2025-12-05 20:01:18 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: ea1a3343-8b33-5b00-84c4-85bfc920a4eb with identifier: 1788780
2025-12-05 20:01:18 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: ea1a3343-8b33-5b00-84c4-85bfc920a4eb with identifier: 1788780
2025-12-05 20:01:18 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: ea1a3343-8b33-5b00-84c4-85bfc920a4eb with identifier: 1788780 to the API
2025-12-05 20:01:18 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:18 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:18 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/247/0654d8a66fb249ebaf9df93bc71728da/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 109,
 'color': 'Negro',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.6L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'ea1a3343-8b33-5b00-84c4-85bfc920a4eb',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'PASSAT',
 'odometer': 142375,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ea1a3343-8b33-5b00-84c4-85bfc920a4eb/ea1a3343-8b33-5b00-84c4-85bfc920a4eb_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 358194,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/247/0654d8a66fb249ebaf9df93bc71728da/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-11-20',
 'seats_number': 5,
 'seller_name': None,
 'title': 'VOLKSWAGEN PASSAT 1.6TDI 88kW (120CV) CONNECT SCR DSG',
 'transmission': 'manual',
 'trim': '1.6TDI 88kW (120CV) CONNECT SCR DSG',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788780%27 HTTP/1.1" 200 None
2025-12-05 20:01:18 [ayvens.es] INFO: Saving data for 1788780: {'created_time': 1764964878.387973, 'last_price_update_time': 1764964878.387984, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788780') HTTP/1.1" 204 0
2025-12-05 20:01:18 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/809/91ff773b16544485902f3995d64b59fa/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:18 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/331/12ca9c587e2e4987bdbad681e08c30c5/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:18 [ayvens.es] INFO: Found listing with ID: 1788791
2025-12-05 20:01:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788791%27 HTTP/1.1" 200 None
2025-12-05 20:01:18 [ayvens.es] INFO: Scrape type for 1788791: 1
2025-12-05 20:01:18 [ayvens.es] INFO: Extracting data for car ID: 1788791
2025-12-05 20:01:18 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: ac3ed31c-527f-5cd8-bb6c-69c209e903e0 with identifier: 1788791
2025-12-05 20:01:18 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ac3ed31c-527f-5cd8-bb6c-69c209e903e0 with scrape type 1
2025-12-05 20:01:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item ac3ed31c-527f-5cd8-bb6c-69c209e903e0
2025-12-05 20:01:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F467%252F984b5609328044f29e68d5b514ac6c51%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/467/984b5609328044f29e68d5b514ac6c51/images/large/1.jpg not downloaded yet
2025-12-05 20:01:18 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:18 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:18 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:18 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a70290>
2025-12-05 20:01:18 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:18 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:18 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:18 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:18 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:18 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:18 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_950afb39-23df-5e09-81d2-479bc21e91db/950afb39-23df-5e09-81d2-479bc21e91db_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F232%252F503abfd227df4755bd0662aed6350753%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/232/503abfd227df4755bd0662aed6350753/images/large/1.jpg saved to cache
2025-12-05 20:01:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 950afb39-23df-5e09-81d2-479bc21e91db
2025-12-05 20:01:18 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 950afb39-23df-5e09-81d2-479bc21e91db, skipping ID generation
2025-12-05 20:01:18 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 950afb39-23df-5e09-81d2-479bc21e91db with scrape type 1
2025-12-05 20:01:18 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 950afb39-23df-5e09-81d2-479bc21e91db from tracking dictionaries
2025-12-05 20:01:18 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 950afb39-23df-5e09-81d2-479bc21e91db sending to next pipeline
2025-12-05 20:01:18 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 950afb39-23df-5e09-81d2-479bc21e91db, identifier: 1788783
2025-12-05 20:01:18 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:18 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:18 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:18 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a72290>
2025-12-05 20:01:18 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:18 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:19 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:19 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:19 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:19 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:19 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_e66bf8d3-822f-5e7e-97cf-38eff6b594ea/e66bf8d3-822f-5e7e-97cf-38eff6b594ea_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F797%252Fb4dad849b8934076a422bf7164776e96%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/797/b4dad849b8934076a422bf7164776e96/images/large/1.jpg saved to cache
2025-12-05 20:01:19 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item e66bf8d3-822f-5e7e-97cf-38eff6b594ea
2025-12-05 20:01:19 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: e66bf8d3-822f-5e7e-97cf-38eff6b594ea, skipping ID generation
2025-12-05 20:01:19 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e66bf8d3-822f-5e7e-97cf-38eff6b594ea with scrape type 1
2025-12-05 20:01:19 [crawlers.middlewares.photo_download_middleware] INFO: Removing item e66bf8d3-822f-5e7e-97cf-38eff6b594ea from tracking dictionaries
2025-12-05 20:01:19 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item e66bf8d3-822f-5e7e-97cf-38eff6b594ea sending to next pipeline
2025-12-05 20:01:19 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: e66bf8d3-822f-5e7e-97cf-38eff6b594ea, identifier: 1788782
2025-12-05 20:01:19 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 950afb39-23df-5e09-81d2-479bc21e91db with identifier: 1788783
2025-12-05 20:01:19 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 950afb39-23df-5e09-81d2-479bc21e91db with identifier: 1788783
2025-12-05 20:01:19 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 950afb39-23df-5e09-81d2-479bc21e91db with identifier: 1788783 to the API
2025-12-05 20:01:19 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:19 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:19 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/232/503abfd227df4755bd0662aed6350753/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 76,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.8L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'hybrid',
 'id': '950afb39-23df-5e09-81d2-479bc21e91db',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'TOYOTA',
 'model': 'COROLLA',
 'odometer': 85906,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_950afb39-23df-5e09-81d2-479bc21e91db/950afb39-23df-5e09-81d2-479bc21e91db_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 351848,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/232/503abfd227df4755bd0662aed6350753/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-09-16',
 'seats_number': 5,
 'seller_name': None,
 'title': 'TOYOTA COROLLA 1.8 HSD Aire TS',
 'transmission': 'manual',
 'trim': '1.8 HSD Aire TS',
 'vin': None,
 'year': 2019}
2025-12-05 20:01:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788783%27 HTTP/1.1" 200 None
2025-12-05 20:01:19 [ayvens.es] INFO: Saving data for 1788783: {'created_time': 1764964879.683121, 'last_price_update_time': 1764964879.683131, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788783') HTTP/1.1" 204 0
2025-12-05 20:01:19 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: e66bf8d3-822f-5e7e-97cf-38eff6b594ea with identifier: 1788782
2025-12-05 20:01:19 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: e66bf8d3-822f-5e7e-97cf-38eff6b594ea with identifier: 1788782
2025-12-05 20:01:19 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'bmw', 'model': 'x1', 'trim': 'sdrive16d', 'title': 'bmw x1 sdrive16d', 'price': 0, 'odometer': 210886, 'odometer_reading_unit': 'km', 'registration_date': '2019-10-29', 'year': 2019, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.5l', 'c_o_2_emission_value': 118, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'e66bf8d3-822f-5e7e-97cf-38eff6b594ea', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_e66bf8d3-822f-5e7e-97cf-38eff6b594ea/e66bf8d3-822f-5e7e-97cf-38eff6b594ea_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/797/b4dad849b8934076a422bf7164776e96/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 329948}], 'current_year_diff': 6, 'age_in_months': 73, 'make_model': 'bmw x1', 'sanitized_make': 'bmw', 'odometer_in_km': 210886}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 73,
     'auction_title': '',
     'c_o_2_emission_value': 118,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 6,
     'doors_number': 5,
     'engine': '1.5l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': 'e66bf8d3-822f-5e7e-97cf-38eff6b594ea',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'bmw',
     'make_model': 'bmw x1',
     'model': 'x1',
     'odometer': 210886,
     'odometer_in_km': 210886,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_e66bf8d3-822f-5e7e-97cf-38eff6b594ea/e66bf8d3-822f-5e7e-97cf-38eff6b594ea_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 329948,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/797/b4dad849b8934076a422bf7164776e96/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2019-10-29',
     'sanitized_make': 'bmw',
     'seats_number': 5,
     'seller_name': None,
     'title': 'bmw x1 sdrive16d',
     'transmission': 'manual',
     'trim': 'sdrive16d',
     'vin': None,
     'year': 2019}
2025-12-05 20:01:19 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE failed for item: e66bf8d3-822f-5e7e-97cf-38eff6b594ea spider: ayvens.es
2025-12-05 20:01:20 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: e66bf8d3-822f-5e7e-97cf-38eff6b594ea due to rule: high_mileage.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'bmw', 'model': 'x1', 'trim': 'sdrive16d', 'title': 'bmw x1 sdrive16d', 'price': 0, 'odometer': 210886, 'odometer_reading_unit': 'km', 'registration_date': '2019-10-29', 'year': 2019, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.5l', 'c_o_2_emission_value': 118, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'e66bf8d3-822f-5e7e-97cf-38eff6b594ea', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_e66bf8d3-822f-5e7e-97cf-38eff6b594ea/e66bf8d3-822f-5e7e-97cf-38eff6b594ea_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/797/b4dad849b8934076a422bf7164776e96/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 329948}], 'current_year_diff': 6, 'age_in_months': 73, 'make_model': 'bmw x1', 'sanitized_make': 'bmw', 'odometer_in_km': 210886}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 73,
     'auction_title': '',
     'c_o_2_emission_value': 118,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 6,
     'doors_number': 5,
     'engine': '1.5l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': 'e66bf8d3-822f-5e7e-97cf-38eff6b594ea',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'bmw',
     'make_model': 'bmw x1',
     'model': 'x1',
     'odometer': 210886,
     'odometer_in_km': 210886,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_e66bf8d3-822f-5e7e-97cf-38eff6b594ea/e66bf8d3-822f-5e7e-97cf-38eff6b594ea_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 329948,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/797/b4dad849b8934076a422bf7164776e96/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2019-10-29',
     'sanitized_make': 'bmw',
     'seats_number': 5,
     'seller_name': None,
     'title': 'bmw x1 sdrive16d',
     'transmission': 'manual',
     'trim': 'sdrive16d',
     'vin': None,
     'year': 2019}
{'auction_title': '',
 'c_o_2_emission_value': 118,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'e66bf8d3-822f-5e7e-97cf-38eff6b594ea',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'BMW',
 'model': 'X1',
 'odometer': 210886,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_e66bf8d3-822f-5e7e-97cf-38eff6b594ea/e66bf8d3-822f-5e7e-97cf-38eff6b594ea_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 329948,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/797/b4dad849b8934076a422bf7164776e96/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-10-29',
 'seats_number': 5,
 'seller_name': None,
 'title': 'BMW X1 sDrive16d',
 'transmission': 'manual',
 'trim': 'sDrive16d',
 'vin': None,
 'year': 2019}
2025-12-05 20:01:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788782%27 HTTP/1.1" 200 None
2025-12-05 20:01:20 [ayvens.es] INFO: Saving data for 1788782: {'created_time': 1764964880.07418, 'last_price_update_time': 1764964880.07419, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788782') HTTP/1.1" 204 0
2025-12-05 20:01:20 [ayvens.es] INFO: Found listing with ID: 1788792
2025-12-05 20:01:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788792%27 HTTP/1.1" 200 None
2025-12-05 20:01:20 [ayvens.es] INFO: Scrape type for 1788792: 1
2025-12-05 20:01:20 [ayvens.es] INFO: Extracting data for car ID: 1788792
2025-12-05 20:01:20 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: f0af4752-b7e0-5ed4-8c60-463dda6072ad with identifier: 1788792
2025-12-05 20:01:20 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f0af4752-b7e0-5ed4-8c60-463dda6072ad with scrape type 1
2025-12-05 20:01:20 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item f0af4752-b7e0-5ed4-8c60-463dda6072ad
2025-12-05 20:01:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F448%252F63342e991de34060ada7a84465c93dbe%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/448/63342e991de34060ada7a84465c93dbe/images/large/1.jpg not downloaded yet
2025-12-05 20:01:20 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:20 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:20 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:20 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b05de750>
2025-12-05 20:01:20 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:20 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:20 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:01:20 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:20 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:20 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:20 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_db150b0c-38e8-5e79-a618-d63d29fcc23b/db150b0c-38e8-5e79-a618-d63d29fcc23b_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F809%252F91ff773b16544485902f3995d64b59fa%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/809/91ff773b16544485902f3995d64b59fa/images/large/1.jpg saved to cache
2025-12-05 20:01:20 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item db150b0c-38e8-5e79-a618-d63d29fcc23b
2025-12-05 20:01:20 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: db150b0c-38e8-5e79-a618-d63d29fcc23b, skipping ID generation
2025-12-05 20:01:20 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item db150b0c-38e8-5e79-a618-d63d29fcc23b with scrape type 1
2025-12-05 20:01:20 [crawlers.middlewares.photo_download_middleware] INFO: Removing item db150b0c-38e8-5e79-a618-d63d29fcc23b from tracking dictionaries
2025-12-05 20:01:20 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item db150b0c-38e8-5e79-a618-d63d29fcc23b sending to next pipeline
2025-12-05 20:01:20 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: db150b0c-38e8-5e79-a618-d63d29fcc23b, identifier: 1788784
2025-12-05 20:01:20 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:20 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:20 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:20 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b4c310>
2025-12-05 20:01:20 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:20 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:20 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:01:20 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:20 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:20 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:20 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_6894e7db-d32e-5705-bee1-ad3ca9ee3b32/6894e7db-d32e-5705-bee1-ad3ca9ee3b32_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F331%252F12ca9c587e2e4987bdbad681e08c30c5%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/331/12ca9c587e2e4987bdbad681e08c30c5/images/large/1.jpg saved to cache
2025-12-05 20:01:20 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 6894e7db-d32e-5705-bee1-ad3ca9ee3b32
2025-12-05 20:01:20 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 6894e7db-d32e-5705-bee1-ad3ca9ee3b32, skipping ID generation
2025-12-05 20:01:20 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6894e7db-d32e-5705-bee1-ad3ca9ee3b32 with scrape type 1
2025-12-05 20:01:20 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 6894e7db-d32e-5705-bee1-ad3ca9ee3b32 from tracking dictionaries
2025-12-05 20:01:20 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 6894e7db-d32e-5705-bee1-ad3ca9ee3b32 sending to next pipeline
2025-12-05 20:01:21 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 6894e7db-d32e-5705-bee1-ad3ca9ee3b32, identifier: 1788785
2025-12-05 20:01:21 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/086/70cb875d589d4d87a16bca97f36ed74d/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:21 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/738/8a930f8abd15444e8e4e28a61ca938ad/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:21 [ayvens.es] INFO: Found listing with ID: 1788793
2025-12-05 20:01:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788793%27 HTTP/1.1" 200 None
2025-12-05 20:01:21 [ayvens.es] INFO: Scrape type for 1788793: 1
2025-12-05 20:01:21 [ayvens.es] INFO: Extracting data for car ID: 1788793
2025-12-05 20:01:21 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: ea07288e-6ac6-5f75-a7f7-02cb3e1b7182 with identifier: 1788793
2025-12-05 20:01:21 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ea07288e-6ac6-5f75-a7f7-02cb3e1b7182 with scrape type 1
2025-12-05 20:01:21 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item ea07288e-6ac6-5f75-a7f7-02cb3e1b7182
2025-12-05 20:01:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F995%252F72f1a217b7f84192835f843f8272ba31%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/995/72f1a217b7f84192835f843f8272ba31/images/large/1.jpg not downloaded yet
2025-12-05 20:01:21 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: db150b0c-38e8-5e79-a618-d63d29fcc23b with identifier: 1788784
2025-12-05 20:01:21 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: db150b0c-38e8-5e79-a618-d63d29fcc23b with identifier: 1788784
2025-12-05 20:01:21 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: db150b0c-38e8-5e79-a618-d63d29fcc23b with identifier: 1788784 to the API
2025-12-05 20:01:21 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:21 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:21 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/809/91ff773b16544485902f3995d64b59fa/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 102,
 'color': 'Rojo',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.8L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'hybrid',
 'id': 'db150b0c-38e8-5e79-a618-d63d29fcc23b',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'TOYOTA',
 'model': 'COROLLA',
 'odometer': 69138,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_db150b0c-38e8-5e79-a618-d63d29fcc23b/db150b0c-38e8-5e79-a618-d63d29fcc23b_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 380206,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/809/91ff773b16544485902f3995d64b59fa/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-06-22',
 'seats_number': 5,
 'seller_name': None,
 'title': 'TOYOTA COROLLA 1.8 HSD Alisio',
 'transmission': 'manual',
 'trim': '1.8 HSD Alisio',
 'vin': None,
 'year': 2021}
2025-12-05 20:01:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788784%27 HTTP/1.1" 200 None
2025-12-05 20:01:21 [ayvens.es] INFO: Saving data for 1788784: {'created_time': 1764964881.379669, 'last_price_update_time': 1764964881.379677, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788784') HTTP/1.1" 204 0
2025-12-05 20:01:21 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 6894e7db-d32e-5705-bee1-ad3ca9ee3b32 with identifier: 1788785
2025-12-05 20:01:21 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 6894e7db-d32e-5705-bee1-ad3ca9ee3b32 with identifier: 1788785
2025-12-05 20:01:21 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'toyota', 'model': 'yaris', 'trim': '1.5 120h business plus', 'title': 'toyota yaris 1.5 120h business plus', 'price': 0, 'odometer': 228220, 'odometer_reading_unit': 'km', 'registration_date': '2021-02-25', 'year': 2021, 'fuel_type': 'hybrid', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.5l', 'c_o_2_emission_value': 88, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '6894e7db-d32e-5705-bee1-ad3ca9ee3b32', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_6894e7db-d32e-5705-bee1-ad3ca9ee3b32/6894e7db-d32e-5705-bee1-ad3ca9ee3b32_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/331/12ca9c587e2e4987bdbad681e08c30c5/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 357776}], 'current_year_diff': 4, 'age_in_months': 57, 'make_model': 'toyota yaris', 'sanitized_make': 'toyota', 'odometer_in_km': 228220}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 57,
     'auction_title': '',
     'c_o_2_emission_value': 88,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 4,
     'doors_number': 5,
     'engine': '1.5l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'hybrid',
     'id': '6894e7db-d32e-5705-bee1-ad3ca9ee3b32',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'toyota',
     'make_model': 'toyota yaris',
     'model': 'yaris',
     'odometer': 228220,
     'odometer_in_km': 228220,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_6894e7db-d32e-5705-bee1-ad3ca9ee3b32/6894e7db-d32e-5705-bee1-ad3ca9ee3b32_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 357776,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/331/12ca9c587e2e4987bdbad681e08c30c5/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2021-02-25',
     'sanitized_make': 'toyota',
     'seats_number': 5,
     'seller_name': None,
     'title': 'toyota yaris 1.5 120h business plus',
     'transmission': 'manual',
     'trim': '1.5 120h business plus',
     'vin': None,
     'year': 2021}
2025-12-05 20:01:21 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE failed for item: 6894e7db-d32e-5705-bee1-ad3ca9ee3b32 spider: ayvens.es
2025-12-05 20:01:21 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 6894e7db-d32e-5705-bee1-ad3ca9ee3b32 due to rule: high_mileage.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'toyota', 'model': 'yaris', 'trim': '1.5 120h business plus', 'title': 'toyota yaris 1.5 120h business plus', 'price': 0, 'odometer': 228220, 'odometer_reading_unit': 'km', 'registration_date': '2021-02-25', 'year': 2021, 'fuel_type': 'hybrid', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.5l', 'c_o_2_emission_value': 88, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '6894e7db-d32e-5705-bee1-ad3ca9ee3b32', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_6894e7db-d32e-5705-bee1-ad3ca9ee3b32/6894e7db-d32e-5705-bee1-ad3ca9ee3b32_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/331/12ca9c587e2e4987bdbad681e08c30c5/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 357776}], 'current_year_diff': 4, 'age_in_months': 57, 'make_model': 'toyota yaris', 'sanitized_make': 'toyota', 'odometer_in_km': 228220}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 57,
     'auction_title': '',
     'c_o_2_emission_value': 88,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 4,
     'doors_number': 5,
     'engine': '1.5l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'hybrid',
     'id': '6894e7db-d32e-5705-bee1-ad3ca9ee3b32',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'toyota',
     'make_model': 'toyota yaris',
     'model': 'yaris',
     'odometer': 228220,
     'odometer_in_km': 228220,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_6894e7db-d32e-5705-bee1-ad3ca9ee3b32/6894e7db-d32e-5705-bee1-ad3ca9ee3b32_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 357776,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/331/12ca9c587e2e4987bdbad681e08c30c5/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2021-02-25',
     'sanitized_make': 'toyota',
     'seats_number': 5,
     'seller_name': None,
     'title': 'toyota yaris 1.5 120h business plus',
     'transmission': 'manual',
     'trim': '1.5 120h business plus',
     'vin': None,
     'year': 2021}
{'auction_title': '',
 'c_o_2_emission_value': 88,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'hybrid',
 'id': '6894e7db-d32e-5705-bee1-ad3ca9ee3b32',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'TOYOTA',
 'model': 'YARIS',
 'odometer': 228220,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_6894e7db-d32e-5705-bee1-ad3ca9ee3b32/6894e7db-d32e-5705-bee1-ad3ca9ee3b32_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 357776,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/331/12ca9c587e2e4987bdbad681e08c30c5/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-02-25',
 'seats_number': 5,
 'seller_name': None,
 'title': 'TOYOTA YARIS 1.5 120H Business Plus',
 'transmission': 'manual',
 'trim': '1.5 120H Business Plus',
 'vin': None,
 'year': 2021}
2025-12-05 20:01:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788785%27 HTTP/1.1" 200 None
2025-12-05 20:01:21 [ayvens.es] INFO: Saving data for 1788785: {'created_time': 1764964881.613205, 'last_price_update_time': 1764964881.613217, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788785') HTTP/1.1" 204 0
2025-12-05 20:01:21 [ayvens.es] INFO: Found listing with ID: 1788794
2025-12-05 20:01:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788794%27 HTTP/1.1" 200 None
2025-12-05 20:01:21 [ayvens.es] INFO: Scrape type for 1788794: 1
2025-12-05 20:01:21 [ayvens.es] INFO: Extracting data for car ID: 1788794
2025-12-05 20:01:21 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 21edc391-54ca-5273-bb54-7ac17071eea9 with identifier: 1788794
2025-12-05 20:01:21 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 21edc391-54ca-5273-bb54-7ac17071eea9 with scrape type 1
2025-12-05 20:01:21 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 21edc391-54ca-5273-bb54-7ac17071eea9
2025-12-05 20:01:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F540%252F8630530a1353470e91e55b8237bb6df2%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/540/8630530a1353470e91e55b8237bb6df2/images/large/1.jpg not downloaded yet
2025-12-05 20:01:21 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/604/ba1fa4a70626424a88178d1b65a3ed97/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:21 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/751/886bd7e5a8c948448d173199c52b1f40/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:21 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:21 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:21 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:21 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0548990>
2025-12-05 20:01:21 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:21 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:21 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:21 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:21 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:21 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:22 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_5ff4b563-d73a-5f7a-85d7-97c0d5c7dbdb/5ff4b563-d73a-5f7a-85d7-97c0d5c7dbdb_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F086%252F70cb875d589d4d87a16bca97f36ed74d%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/086/70cb875d589d4d87a16bca97f36ed74d/images/large/1.jpg saved to cache
2025-12-05 20:01:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 5ff4b563-d73a-5f7a-85d7-97c0d5c7dbdb
2025-12-05 20:01:22 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 5ff4b563-d73a-5f7a-85d7-97c0d5c7dbdb, skipping ID generation
2025-12-05 20:01:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 5ff4b563-d73a-5f7a-85d7-97c0d5c7dbdb with scrape type 1
2025-12-05 20:01:22 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 5ff4b563-d73a-5f7a-85d7-97c0d5c7dbdb from tracking dictionaries
2025-12-05 20:01:22 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 5ff4b563-d73a-5f7a-85d7-97c0d5c7dbdb sending to next pipeline
2025-12-05 20:01:22 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 5ff4b563-d73a-5f7a-85d7-97c0d5c7dbdb, identifier: 1788786
2025-12-05 20:01:22 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:22 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:22 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:22 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0471ed0>
2025-12-05 20:01:22 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:22 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:22 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:22 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:22 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:22 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:22 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_c290bec7-2c63-5f76-9426-c6705be56ab0/c290bec7-2c63-5f76-9426-c6705be56ab0_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F738%252F8a930f8abd15444e8e4e28a61ca938ad%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/738/8a930f8abd15444e8e4e28a61ca938ad/images/large/1.jpg saved to cache
2025-12-05 20:01:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item c290bec7-2c63-5f76-9426-c6705be56ab0
2025-12-05 20:01:22 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: c290bec7-2c63-5f76-9426-c6705be56ab0, skipping ID generation
2025-12-05 20:01:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item c290bec7-2c63-5f76-9426-c6705be56ab0 with scrape type 1
2025-12-05 20:01:22 [crawlers.middlewares.photo_download_middleware] INFO: Removing item c290bec7-2c63-5f76-9426-c6705be56ab0 from tracking dictionaries
2025-12-05 20:01:22 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item c290bec7-2c63-5f76-9426-c6705be56ab0 sending to next pipeline
2025-12-05 20:01:22 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: c290bec7-2c63-5f76-9426-c6705be56ab0, identifier: 1788787
2025-12-05 20:01:22 [ayvens.es] INFO: Found listing with ID: 1788795
2025-12-05 20:01:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788795%27 HTTP/1.1" 200 None
2025-12-05 20:01:22 [ayvens.es] INFO: Scrape type for 1788795: 1
2025-12-05 20:01:22 [ayvens.es] INFO: Extracting data for car ID: 1788795
2025-12-05 20:01:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 680aef6e-7cb6-59d7-aed9-0b805dfe5cd2 with identifier: 1788795
2025-12-05 20:01:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 680aef6e-7cb6-59d7-aed9-0b805dfe5cd2 with scrape type 1
2025-12-05 20:01:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 680aef6e-7cb6-59d7-aed9-0b805dfe5cd2
2025-12-05 20:01:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F732%252F1d6760db1a584956b89ef34d22e314f5%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/732/1d6760db1a584956b89ef34d22e314f5/images/large/1.jpg not downloaded yet
2025-12-05 20:01:22 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 5ff4b563-d73a-5f7a-85d7-97c0d5c7dbdb with identifier: 1788786
2025-12-05 20:01:22 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 5ff4b563-d73a-5f7a-85d7-97c0d5c7dbdb with identifier: 1788786
2025-12-05 20:01:22 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'volkswagen' containing 'volkswagen' and model 'caddy' containing 'caddy'
2025-12-05 20:01:22 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 5ff4b563-d73a-5f7a-85d7-97c0d5c7dbdb spider: ayvens.es
2025-12-05 20:01:22 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 5ff4b563-d73a-5f7a-85d7-97c0d5c7dbdb due to rule: makes_models_not_allowed.json, error: Rejected due to make 'volkswagen' containing 'volkswagen' and model 'caddy' containing 'caddy'
{'auction_title': '',
 'c_o_2_emission_value': 127,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.4L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'gas',
 'id': '5ff4b563-d73a-5f7a-85d7-97c0d5c7dbdb',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'CADDY',
 'odometer': 89069,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_5ff4b563-d73a-5f7a-85d7-97c0d5c7dbdb/5ff4b563-d73a-5f7a-85d7-97c0d5c7dbdb_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 327660,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/086/70cb875d589d4d87a16bca97f36ed74d/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-06-12',
 'seats_number': 2,
 'seller_name': None,
 'title': 'VOLKSWAGEN CADDY Profesional Furgon 1.4 TGI 81kW BM',
 'transmission': 'manual',
 'trim': 'Profesional Furgon 1.4 TGI 81kW BM',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788786%27 HTTP/1.1" 200 None
2025-12-05 20:01:22 [ayvens.es] INFO: Saving data for 1788786: {'created_time': 1764964882.977067, 'last_price_update_time': 1764964882.977077, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788786') HTTP/1.1" 204 0
2025-12-05 20:01:23 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: c290bec7-2c63-5f76-9426-c6705be56ab0 with identifier: 1788787
2025-12-05 20:01:23 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: c290bec7-2c63-5f76-9426-c6705be56ab0 with identifier: 1788787
2025-12-05 20:01:23 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: c290bec7-2c63-5f76-9426-c6705be56ab0 with identifier: 1788787 to the API
2025-12-05 20:01:23 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:23 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:23 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/738/8a930f8abd15444e8e4e28a61ca938ad/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 89,
 'color': 'Azul',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'hybrid',
 'id': 'c290bec7-2c63-5f76-9426-c6705be56ab0',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'TOYOTA',
 'model': 'COROLLA',
 'odometer': 42765,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_c290bec7-2c63-5f76-9426-c6705be56ab0/c290bec7-2c63-5f76-9426-c6705be56ab0_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 354901,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/738/8a930f8abd15444e8e4e28a61ca938ad/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-09-02',
 'seats_number': 5,
 'seller_name': None,
 'title': 'TOYOTA COROLLA 2.0 180H ADVANCE E-CVT',
 'transmission': 'manual',
 'trim': '2.0 180H ADVANCE E-CVT',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788787%27 HTTP/1.1" 200 None
2025-12-05 20:01:23 [ayvens.es] INFO: Saving data for 1788787: {'created_time': 1764964883.404161, 'last_price_update_time': 1764964883.404172, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788787') HTTP/1.1" 204 0
2025-12-05 20:01:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/080/67166f8b49d04340a54226538ee6ecf3/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:23 [ayvens.es] INFO: Found listing with ID: 1788796
2025-12-05 20:01:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788796%27 HTTP/1.1" 200 None
2025-12-05 20:01:23 [ayvens.es] INFO: Scrape type for 1788796: 1
2025-12-05 20:01:23 [ayvens.es] INFO: Extracting data for car ID: 1788796
2025-12-05 20:01:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: fb158477-27d7-589e-acb6-4ea177572fac with identifier: 1788796
2025-12-05 20:01:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item fb158477-27d7-589e-acb6-4ea177572fac with scrape type 1
2025-12-05 20:01:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item fb158477-27d7-589e-acb6-4ea177572fac
2025-12-05 20:01:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F169%252F21e4f4b71cbd4319954ab2121913ca22%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/169/21e4f4b71cbd4319954ab2121913ca22/images/large/1.jpg not downloaded yet
2025-12-05 20:01:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/467/984b5609328044f29e68d5b514ac6c51/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:23 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:23 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:23 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:23 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0456590>
2025-12-05 20:01:23 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:23 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:23 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:01:23 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:23 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:23 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:23 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_fe518f07-ed7e-5596-a2bd-38018f32eadf/fe518f07-ed7e-5596-a2bd-38018f32eadf_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F604%252Fba1fa4a70626424a88178d1b65a3ed97%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/604/ba1fa4a70626424a88178d1b65a3ed97/images/large/1.jpg saved to cache
2025-12-05 20:01:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item fe518f07-ed7e-5596-a2bd-38018f32eadf
2025-12-05 20:01:23 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: fe518f07-ed7e-5596-a2bd-38018f32eadf, skipping ID generation
2025-12-05 20:01:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item fe518f07-ed7e-5596-a2bd-38018f32eadf with scrape type 1
2025-12-05 20:01:23 [crawlers.middlewares.photo_download_middleware] INFO: Removing item fe518f07-ed7e-5596-a2bd-38018f32eadf from tracking dictionaries
2025-12-05 20:01:23 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item fe518f07-ed7e-5596-a2bd-38018f32eadf sending to next pipeline
2025-12-05 20:01:24 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: fe518f07-ed7e-5596-a2bd-38018f32eadf, identifier: 1788788
2025-12-05 20:01:24 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:24 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:24 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:24 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b05276d0>
2025-12-05 20:01:24 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:24 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:24 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:01:24 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:24 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:24 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:24 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_5b91fcd0-6c3f-5c77-b7ff-68d5654d467d/5b91fcd0-6c3f-5c77-b7ff-68d5654d467d_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F751%252F886bd7e5a8c948448d173199c52b1f40%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/751/886bd7e5a8c948448d173199c52b1f40/images/large/1.jpg saved to cache
2025-12-05 20:01:24 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 5b91fcd0-6c3f-5c77-b7ff-68d5654d467d
2025-12-05 20:01:24 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 5b91fcd0-6c3f-5c77-b7ff-68d5654d467d, skipping ID generation
2025-12-05 20:01:24 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 5b91fcd0-6c3f-5c77-b7ff-68d5654d467d with scrape type 1
2025-12-05 20:01:24 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 5b91fcd0-6c3f-5c77-b7ff-68d5654d467d from tracking dictionaries
2025-12-05 20:01:24 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 5b91fcd0-6c3f-5c77-b7ff-68d5654d467d sending to next pipeline
2025-12-05 20:01:24 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 5b91fcd0-6c3f-5c77-b7ff-68d5654d467d, identifier: 1788789
2025-12-05 20:01:24 [ayvens.es] INFO: Found listing with ID: 1788797
2025-12-05 20:01:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788797%27 HTTP/1.1" 200 None
2025-12-05 20:01:24 [ayvens.es] INFO: Scrape type for 1788797: 1
2025-12-05 20:01:24 [ayvens.es] INFO: Extracting data for car ID: 1788797
2025-12-05 20:01:24 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 964ef805-c965-583b-b118-cc1075504436 with identifier: 1788797
2025-12-05 20:01:24 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 964ef805-c965-583b-b118-cc1075504436 with scrape type 1
2025-12-05 20:01:24 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 964ef805-c965-583b-b118-cc1075504436
2025-12-05 20:01:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F393%252F2ceccb4749f14a008f6abb24d74704fa%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/393/2ceccb4749f14a008f6abb24d74704fa/images/large/1.jpg not downloaded yet
2025-12-05 20:01:24 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: fe518f07-ed7e-5596-a2bd-38018f32eadf with identifier: 1788788
2025-12-05 20:01:24 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:01:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:01:24 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: fe518f07-ed7e-5596-a2bd-38018f32eadf with identifier: 1788788
2025-12-05 20:01:24 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: fe518f07-ed7e-5596-a2bd-38018f32eadf with identifier: 1788788 to the API
2025-12-05 20:01:24 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:24 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:24 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/604/ba1fa4a70626424a88178d1b65a3ed97/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 127,
 'color': 'Polar',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.6L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'hybrid',
 'id': 'fe518f07-ed7e-5596-a2bd-38018f32eadf',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'HYUNDAI',
 'model': 'TUCSON',
 'odometer': 50306,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_fe518f07-ed7e-5596-a2bd-38018f32eadf/fe518f07-ed7e-5596-a2bd-38018f32eadf_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 342258,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/604/ba1fa4a70626424a88178d1b65a3ed97/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-08-31',
 'seats_number': 5,
 'seller_name': None,
 'title': 'HYUNDAI TUCSON 1.6 TGDI 169kW (230CV) HEV Maxx Sky Auto',
 'transmission': 'manual',
 'trim': '1.6 TGDI 169kW (230CV) HEV Maxx Sky Auto',
 'vin': None,
 'year': 2021}
2025-12-05 20:01:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788788%27 HTTP/1.1" 200 None
2025-12-05 20:01:24 [ayvens.es] INFO: Saving data for 1788788: {'created_time': 1764964884.88534, 'last_price_update_time': 1764964884.885351, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788788') HTTP/1.1" 204 0
2025-12-05 20:01:24 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 5b91fcd0-6c3f-5c77-b7ff-68d5654d467d with identifier: 1788789
2025-12-05 20:01:24 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 5b91fcd0-6c3f-5c77-b7ff-68d5654d467d with identifier: 1788789
2025-12-05 20:01:25 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 5b91fcd0-6c3f-5c77-b7ff-68d5654d467d with identifier: 1788789 to the API
2025-12-05 20:01:25 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:25 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:25 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/751/886bd7e5a8c948448d173199c52b1f40/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 139,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.4L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'gas',
 'id': '5b91fcd0-6c3f-5c77-b7ff-68d5654d467d',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'OPEL',
 'model': 'MOKKA X',
 'odometer': 92250,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_5b91fcd0-6c3f-5c77-b7ff-68d5654d467d/5b91fcd0-6c3f-5c77-b7ff-68d5654d467d_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 337797,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/751/886bd7e5a8c948448d173199c52b1f40/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-06-07',
 'seats_number': 5,
 'seller_name': None,
 'title': 'OPEL MOKKA X 1.4 T 103kW GLP 4X2 Design Line',
 'transmission': 'manual',
 'trim': '1.4 T 103kW GLP 4X2 Design Line',
 'vin': None,
 'year': 2019}
2025-12-05 20:01:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788789%27 HTTP/1.1" 200 None
2025-12-05 20:01:25 [ayvens.es] INFO: Saving data for 1788789: {'created_time': 1764964885.301503, 'last_price_update_time': 1764964885.301511, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788789') HTTP/1.1" 204 0
2025-12-05 20:01:25 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/995/72f1a217b7f84192835f843f8272ba31/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:25 [ayvens.es] INFO: Found listing with ID: 1788798
2025-12-05 20:01:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788798%27 HTTP/1.1" 200 None
2025-12-05 20:01:25 [ayvens.es] INFO: Scrape type for 1788798: 1
2025-12-05 20:01:25 [ayvens.es] INFO: Extracting data for car ID: 1788798
2025-12-05 20:01:25 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: ee87e13e-1ec9-5552-b595-16f2dcd8af4d with identifier: 1788798
2025-12-05 20:01:25 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ee87e13e-1ec9-5552-b595-16f2dcd8af4d with scrape type 1
2025-12-05 20:01:25 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item ee87e13e-1ec9-5552-b595-16f2dcd8af4d
2025-12-05 20:01:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F177%252F6d1fb6202489434088daad4478fdc3f2%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/177/6d1fb6202489434088daad4478fdc3f2/images/large/1.jpg not downloaded yet
2025-12-05 20:01:25 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:25 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:25 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:25 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a41810>
2025-12-05 20:01:25 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:25 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:25 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:25 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:25 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:25 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:25 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_19941e70-7776-5c54-9017-3f86e20a4ae4/19941e70-7776-5c54-9017-3f86e20a4ae4_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F080%252F67166f8b49d04340a54226538ee6ecf3%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/080/67166f8b49d04340a54226538ee6ecf3/images/large/1.jpg saved to cache
2025-12-05 20:01:25 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 19941e70-7776-5c54-9017-3f86e20a4ae4
2025-12-05 20:01:25 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 19941e70-7776-5c54-9017-3f86e20a4ae4, skipping ID generation
2025-12-05 20:01:25 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 19941e70-7776-5c54-9017-3f86e20a4ae4 with scrape type 1
2025-12-05 20:01:25 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 19941e70-7776-5c54-9017-3f86e20a4ae4 from tracking dictionaries
2025-12-05 20:01:25 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 19941e70-7776-5c54-9017-3f86e20a4ae4 sending to next pipeline
2025-12-05 20:01:25 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 19941e70-7776-5c54-9017-3f86e20a4ae4, identifier: 1788790
2025-12-05 20:01:25 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:25 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:25 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:25 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b04b7bd0>
2025-12-05 20:01:25 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:25 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:26 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:26 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:26 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:26 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:26 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_ac3ed31c-527f-5cd8-bb6c-69c209e903e0/ac3ed31c-527f-5cd8-bb6c-69c209e903e0_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F467%252F984b5609328044f29e68d5b514ac6c51%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/467/984b5609328044f29e68d5b514ac6c51/images/large/1.jpg saved to cache
2025-12-05 20:01:26 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item ac3ed31c-527f-5cd8-bb6c-69c209e903e0
2025-12-05 20:01:26 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: ac3ed31c-527f-5cd8-bb6c-69c209e903e0, skipping ID generation
2025-12-05 20:01:26 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ac3ed31c-527f-5cd8-bb6c-69c209e903e0 with scrape type 1
2025-12-05 20:01:26 [crawlers.middlewares.photo_download_middleware] INFO: Removing item ac3ed31c-527f-5cd8-bb6c-69c209e903e0 from tracking dictionaries
2025-12-05 20:01:26 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item ac3ed31c-527f-5cd8-bb6c-69c209e903e0 sending to next pipeline
2025-12-05 20:01:26 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: ac3ed31c-527f-5cd8-bb6c-69c209e903e0, identifier: 1788791
2025-12-05 20:01:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/448/63342e991de34060ada7a84465c93dbe/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:26 [ayvens.es] INFO: Found listing with ID: 1788799
2025-12-05 20:01:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788799%27 HTTP/1.1" 200 None
2025-12-05 20:01:26 [ayvens.es] INFO: Scrape type for 1788799: 1
2025-12-05 20:01:26 [ayvens.es] INFO: Extracting data for car ID: 1788799
2025-12-05 20:01:26 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 8aec3a22-a764-54c5-9a7a-8c0f3730c908 with identifier: 1788799
2025-12-05 20:01:26 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 8aec3a22-a764-54c5-9a7a-8c0f3730c908 with scrape type 1
2025-12-05 20:01:26 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 8aec3a22-a764-54c5-9a7a-8c0f3730c908
2025-12-05 20:01:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F548%252F6d38b248165d47e5b8da31b676a49adf%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/548/6d38b248165d47e5b8da31b676a49adf/images/large/1.jpg not downloaded yet
2025-12-05 20:01:26 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 19941e70-7776-5c54-9017-3f86e20a4ae4 with identifier: 1788790
2025-12-05 20:01:26 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:01:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:01:26 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 19941e70-7776-5c54-9017-3f86e20a4ae4 with identifier: 1788790
2025-12-05 20:01:26 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'peugeot' containing 'peugeot' and model 'boxer' containing 'boxer'
2025-12-05 20:01:26 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 19941e70-7776-5c54-9017-3f86e20a4ae4 spider: ayvens.es
2025-12-05 20:01:26 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 19941e70-7776-5c54-9017-3f86e20a4ae4 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'peugeot' containing 'peugeot' and model 'boxer' containing 'boxer'
{'auction_title': '',
 'c_o_2_emission_value': 163,
 'color': 'Banquise',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '19941e70-7776-5c54-9017-3f86e20a4ae4',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': 'BOXER',
 'odometer': 143513,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_19941e70-7776-5c54-9017-3f86e20a4ae4/19941e70-7776-5c54-9017-3f86e20a4ae4_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 332217,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/080/67166f8b49d04340a54226538ee6ecf3/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-04-24',
 'seats_number': 3,
 'seller_name': None,
 'title': 'PEUGEOT BOXER 335 L2H2 BlueHDi 96KW (130CV)',
 'transmission': 'manual',
 'trim': '335 L2H2 BlueHDi 96KW (130CV)',
 'vin': None,
 'year': 2019}
2025-12-05 20:01:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788790%27 HTTP/1.1" 200 None
2025-12-05 20:01:26 [ayvens.es] INFO: Saving data for 1788790: {'created_time': 1764964886.818405, 'last_price_update_time': 1764964886.818414, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788790') HTTP/1.1" 204 0
2025-12-05 20:01:26 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: ac3ed31c-527f-5cd8-bb6c-69c209e903e0 with identifier: 1788791
2025-12-05 20:01:26 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: ac3ed31c-527f-5cd8-bb6c-69c209e903e0 with identifier: 1788791
2025-12-05 20:01:26 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'renault' containing 'renault' and model 'kangoo express' containing 'express'
2025-12-05 20:01:26 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: ac3ed31c-527f-5cd8-bb6c-69c209e903e0 spider: ayvens.es
2025-12-05 20:01:27 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: ac3ed31c-527f-5cd8-bb6c-69c209e903e0 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'renault' containing 'renault' and model 'kangoo express' containing 'express'
{'auction_title': '',
 'c_o_2_emission_value': 117,
 'color': 'Pintura',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'ac3ed31c-527f-5cd8-bb6c-69c209e903e0',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'KANGOO EXPRESS',
 'odometer': 88384,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ac3ed31c-527f-5cd8-bb6c-69c209e903e0/ac3ed31c-527f-5cd8-bb6c-69c209e903e0_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 346556,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/467/984b5609328044f29e68d5b514ac6c51/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-10-08',
 'seats_number': 2,
 'seller_name': None,
 'title': 'RENAULT KANGOO EXPRESS Maxi 2p Blue dCi 70 kW (95CV)',
 'transmission': 'manual',
 'trim': 'Maxi 2p Blue dCi 70 kW (95CV)',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788791%27 HTTP/1.1" 200 None
2025-12-05 20:01:27 [ayvens.es] INFO: Saving data for 1788791: {'created_time': 1764964887.078801, 'last_price_update_time': 1764964887.078813, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788791') HTTP/1.1" 204 0
2025-12-05 20:01:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/540/8630530a1353470e91e55b8237bb6df2/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:27 [ayvens.es] INFO: Found listing with ID: 1788800
2025-12-05 20:01:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788800%27 HTTP/1.1" 200 None
2025-12-05 20:01:27 [ayvens.es] INFO: Scrape type for 1788800: 1
2025-12-05 20:01:27 [ayvens.es] INFO: Extracting data for car ID: 1788800
2025-12-05 20:01:27 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: b9670650-994c-576f-bb79-e8dff1806856 with identifier: 1788800
2025-12-05 20:01:27 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b9670650-994c-576f-bb79-e8dff1806856 with scrape type 1
2025-12-05 20:01:27 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item b9670650-994c-576f-bb79-e8dff1806856
2025-12-05 20:01:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F986%252F4d703f59b2f7414ca01323bcd2d37a7f%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/986/4d703f59b2f7414ca01323bcd2d37a7f/images/large/1.jpg not downloaded yet
2025-12-05 20:01:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/732/1d6760db1a584956b89ef34d22e314f5/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:27 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:27 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:27 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:27 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b59510>
2025-12-05 20:01:27 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:27 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:27 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:27 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:27 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:27 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:27 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_ea07288e-6ac6-5f75-a7f7-02cb3e1b7182/ea07288e-6ac6-5f75-a7f7-02cb3e1b7182_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F995%252F72f1a217b7f84192835f843f8272ba31%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/995/72f1a217b7f84192835f843f8272ba31/images/large/1.jpg saved to cache
2025-12-05 20:01:27 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item ea07288e-6ac6-5f75-a7f7-02cb3e1b7182
2025-12-05 20:01:27 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: ea07288e-6ac6-5f75-a7f7-02cb3e1b7182, skipping ID generation
2025-12-05 20:01:27 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ea07288e-6ac6-5f75-a7f7-02cb3e1b7182 with scrape type 1
2025-12-05 20:01:27 [crawlers.middlewares.photo_download_middleware] INFO: Removing item ea07288e-6ac6-5f75-a7f7-02cb3e1b7182 from tracking dictionaries
2025-12-05 20:01:27 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item ea07288e-6ac6-5f75-a7f7-02cb3e1b7182 sending to next pipeline
2025-12-05 20:01:27 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: ea07288e-6ac6-5f75-a7f7-02cb3e1b7182, identifier: 1788793
2025-12-05 20:01:27 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:27 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:27 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:27 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b04b4910>
2025-12-05 20:01:27 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:27 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:27 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:27 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:27 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:27 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:27 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_f0af4752-b7e0-5ed4-8c60-463dda6072ad/f0af4752-b7e0-5ed4-8c60-463dda6072ad_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F448%252F63342e991de34060ada7a84465c93dbe%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/448/63342e991de34060ada7a84465c93dbe/images/large/1.jpg saved to cache
2025-12-05 20:01:27 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item f0af4752-b7e0-5ed4-8c60-463dda6072ad
2025-12-05 20:01:27 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: f0af4752-b7e0-5ed4-8c60-463dda6072ad, skipping ID generation
2025-12-05 20:01:27 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f0af4752-b7e0-5ed4-8c60-463dda6072ad with scrape type 1
2025-12-05 20:01:27 [crawlers.middlewares.photo_download_middleware] INFO: Removing item f0af4752-b7e0-5ed4-8c60-463dda6072ad from tracking dictionaries
2025-12-05 20:01:27 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item f0af4752-b7e0-5ed4-8c60-463dda6072ad sending to next pipeline
2025-12-05 20:01:28 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: f0af4752-b7e0-5ed4-8c60-463dda6072ad, identifier: 1788792
2025-12-05 20:01:28 [ayvens.es] INFO: Found listing with ID: 1788801
2025-12-05 20:01:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788801%27 HTTP/1.1" 200 None
2025-12-05 20:01:28 [ayvens.es] INFO: Scrape type for 1788801: 1
2025-12-05 20:01:28 [ayvens.es] INFO: Extracting data for car ID: 1788801
2025-12-05 20:01:28 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: a3b68e8b-8606-5ed0-8e32-07be20d1db5c with identifier: 1788801
2025-12-05 20:01:28 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a3b68e8b-8606-5ed0-8e32-07be20d1db5c with scrape type 1
2025-12-05 20:01:28 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item a3b68e8b-8606-5ed0-8e32-07be20d1db5c
2025-12-05 20:01:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F358%252Fbe65dc5152fe4da0891f4abb76e51d27%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/358/be65dc5152fe4da0891f4abb76e51d27/images/large/1.jpg not downloaded yet
2025-12-05 20:01:28 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: ea07288e-6ac6-5f75-a7f7-02cb3e1b7182 with identifier: 1788793
2025-12-05 20:01:28 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: ea07288e-6ac6-5f75-a7f7-02cb3e1b7182 with identifier: 1788793
2025-12-05 20:01:28 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'negro', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'mercedes-benz', 'model': 'a-class', 'trim': 'a 180 d', 'title': 'mercedes-benz a-class a 180 d', 'price': 0, 'odometer': 250416, 'odometer_reading_unit': 'km', 'registration_date': '2019-10-22', 'year': 2019, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.5l', 'c_o_2_emission_value': 111, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'ea07288e-6ac6-5f75-a7f7-02cb3e1b7182', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ea07288e-6ac6-5f75-a7f7-02cb3e1b7182/ea07288e-6ac6-5f75-a7f7-02cb3e1b7182_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/995/72f1a217b7f84192835f843f8272ba31/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 312615}], 'current_year_diff': 6, 'age_in_months': 73, 'make_model': 'mercedes-benz a-class', 'sanitized_make': 'mercedesbenz', 'odometer_in_km': 250416}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 73,
     'auction_title': '',
     'c_o_2_emission_value': 111,
     'color': 'negro',
     'currency': 'eur',
     'current_year_diff': 6,
     'doors_number': 5,
     'engine': '1.5l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': 'ea07288e-6ac6-5f75-a7f7-02cb3e1b7182',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'mercedes-benz',
     'make_model': 'mercedes-benz a-class',
     'model': 'a-class',
     'odometer': 250416,
     'odometer_in_km': 250416,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ea07288e-6ac6-5f75-a7f7-02cb3e1b7182/ea07288e-6ac6-5f75-a7f7-02cb3e1b7182_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 312615,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/995/72f1a217b7f84192835f843f8272ba31/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2019-10-22',
     'sanitized_make': 'mercedesbenz',
     'seats_number': 5,
     'seller_name': None,
     'title': 'mercedes-benz a-class a 180 d',
     'transmission': 'manual',
     'trim': 'a 180 d',
     'vin': None,
     'year': 2019}
2025-12-05 20:01:28 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE failed for item: ea07288e-6ac6-5f75-a7f7-02cb3e1b7182 spider: ayvens.es
2025-12-05 20:01:28 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: ea07288e-6ac6-5f75-a7f7-02cb3e1b7182 due to rule: high_mileage.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'negro', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'mercedes-benz', 'model': 'a-class', 'trim': 'a 180 d', 'title': 'mercedes-benz a-class a 180 d', 'price': 0, 'odometer': 250416, 'odometer_reading_unit': 'km', 'registration_date': '2019-10-22', 'year': 2019, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.5l', 'c_o_2_emission_value': 111, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'ea07288e-6ac6-5f75-a7f7-02cb3e1b7182', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ea07288e-6ac6-5f75-a7f7-02cb3e1b7182/ea07288e-6ac6-5f75-a7f7-02cb3e1b7182_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/995/72f1a217b7f84192835f843f8272ba31/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 312615}], 'current_year_diff': 6, 'age_in_months': 73, 'make_model': 'mercedes-benz a-class', 'sanitized_make': 'mercedesbenz', 'odometer_in_km': 250416}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 73,
     'auction_title': '',
     'c_o_2_emission_value': 111,
     'color': 'negro',
     'currency': 'eur',
     'current_year_diff': 6,
     'doors_number': 5,
     'engine': '1.5l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': 'ea07288e-6ac6-5f75-a7f7-02cb3e1b7182',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'mercedes-benz',
     'make_model': 'mercedes-benz a-class',
     'model': 'a-class',
     'odometer': 250416,
     'odometer_in_km': 250416,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ea07288e-6ac6-5f75-a7f7-02cb3e1b7182/ea07288e-6ac6-5f75-a7f7-02cb3e1b7182_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 312615,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/995/72f1a217b7f84192835f843f8272ba31/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2019-10-22',
     'sanitized_make': 'mercedesbenz',
     'seats_number': 5,
     'seller_name': None,
     'title': 'mercedes-benz a-class a 180 d',
     'transmission': 'manual',
     'trim': 'a 180 d',
     'vin': None,
     'year': 2019}
{'auction_title': '',
 'c_o_2_emission_value': 111,
 'color': 'Negro',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'ea07288e-6ac6-5f75-a7f7-02cb3e1b7182',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'MERCEDES-BENZ',
 'model': 'A-CLASS',
 'odometer': 250416,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ea07288e-6ac6-5f75-a7f7-02cb3e1b7182/ea07288e-6ac6-5f75-a7f7-02cb3e1b7182_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 312615,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/995/72f1a217b7f84192835f843f8272ba31/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-10-22',
 'seats_number': 5,
 'seller_name': None,
 'title': 'MERCEDES-BENZ A-CLASS A 180 d',
 'transmission': 'manual',
 'trim': 'A 180 d',
 'vin': None,
 'year': 2019}
2025-12-05 20:01:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788793%27 HTTP/1.1" 200 None
2025-12-05 20:01:28 [ayvens.es] INFO: Saving data for 1788793: {'created_time': 1764964888.314792, 'last_price_update_time': 1764964888.3148, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788793') HTTP/1.1" 204 0
2025-12-05 20:01:28 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: f0af4752-b7e0-5ed4-8c60-463dda6072ad with identifier: 1788792
2025-12-05 20:01:28 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: f0af4752-b7e0-5ed4-8c60-463dda6072ad with identifier: 1788792
2025-12-05 20:01:28 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'citroën', 'model': 'jumpy', 'trim': 'talla xl bluehdi 120 s&s 6v control', 'title': 'citroën jumpy talla xl bluehdi 120 s&s 6v control', 'price': 0, 'odometer': 238418, 'odometer_reading_unit': 'km', 'registration_date': '2021-09-29', 'year': 2021, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 4, 'seats_number': 2, 'vin': None, 'license_plate': None, 'engine': '2.0l', 'c_o_2_emission_value': 190, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'f0af4752-b7e0-5ed4-8c60-463dda6072ad', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_f0af4752-b7e0-5ed4-8c60-463dda6072ad/f0af4752-b7e0-5ed4-8c60-463dda6072ad_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/448/63342e991de34060ada7a84465c93dbe/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 491568}], 'current_year_diff': 4, 'age_in_months': 50, 'make_model': 'citroën jumpy', 'sanitized_make': 'citron', 'odometer_in_km': 238418}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 50,
     'auction_title': '',
     'c_o_2_emission_value': 190,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 4,
     'doors_number': 4,
     'engine': '2.0l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': 'f0af4752-b7e0-5ed4-8c60-463dda6072ad',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'citroën',
     'make_model': 'citroën jumpy',
     'model': 'jumpy',
     'odometer': 238418,
     'odometer_in_km': 238418,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_f0af4752-b7e0-5ed4-8c60-463dda6072ad/f0af4752-b7e0-5ed4-8c60-463dda6072ad_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 491568,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/448/63342e991de34060ada7a84465c93dbe/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2021-09-29',
     'sanitized_make': 'citron',
     'seats_number': 2,
     'seller_name': None,
     'title': 'citroën jumpy talla xl bluehdi 120 s&s 6v control',
     'transmission': 'manual',
     'trim': 'talla xl bluehdi 120 s&s 6v control',
     'vin': None,
     'year': 2021}
2025-12-05 20:01:28 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE failed for item: f0af4752-b7e0-5ed4-8c60-463dda6072ad spider: ayvens.es
2025-12-05 20:01:28 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: f0af4752-b7e0-5ed4-8c60-463dda6072ad due to rule: high_mileage.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'citroën', 'model': 'jumpy', 'trim': 'talla xl bluehdi 120 s&s 6v control', 'title': 'citroën jumpy talla xl bluehdi 120 s&s 6v control', 'price': 0, 'odometer': 238418, 'odometer_reading_unit': 'km', 'registration_date': '2021-09-29', 'year': 2021, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 4, 'seats_number': 2, 'vin': None, 'license_plate': None, 'engine': '2.0l', 'c_o_2_emission_value': 190, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'f0af4752-b7e0-5ed4-8c60-463dda6072ad', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_f0af4752-b7e0-5ed4-8c60-463dda6072ad/f0af4752-b7e0-5ed4-8c60-463dda6072ad_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/448/63342e991de34060ada7a84465c93dbe/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 491568}], 'current_year_diff': 4, 'age_in_months': 50, 'make_model': 'citroën jumpy', 'sanitized_make': 'citron', 'odometer_in_km': 238418}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 50,
     'auction_title': '',
     'c_o_2_emission_value': 190,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 4,
     'doors_number': 4,
     'engine': '2.0l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': 'f0af4752-b7e0-5ed4-8c60-463dda6072ad',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'citroën',
     'make_model': 'citroën jumpy',
     'model': 'jumpy',
     'odometer': 238418,
     'odometer_in_km': 238418,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_f0af4752-b7e0-5ed4-8c60-463dda6072ad/f0af4752-b7e0-5ed4-8c60-463dda6072ad_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 491568,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/448/63342e991de34060ada7a84465c93dbe/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2021-09-29',
     'sanitized_make': 'citron',
     'seats_number': 2,
     'seller_name': None,
     'title': 'citroën jumpy talla xl bluehdi 120 s&s 6v control',
     'transmission': 'manual',
     'trim': 'talla xl bluehdi 120 s&s 6v control',
     'vin': None,
     'year': 2021}
{'auction_title': '',
 'c_o_2_emission_value': 190,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'f0af4752-b7e0-5ed4-8c60-463dda6072ad',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'CITROËN',
 'model': 'JUMPY',
 'odometer': 238418,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_f0af4752-b7e0-5ed4-8c60-463dda6072ad/f0af4752-b7e0-5ed4-8c60-463dda6072ad_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 491568,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/448/63342e991de34060ada7a84465c93dbe/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-09-29',
 'seats_number': 2,
 'seller_name': None,
 'title': 'CITROËN JUMPY Talla XL BlueHDi 120 S&S 6v Control',
 'transmission': 'manual',
 'trim': 'Talla XL BlueHDi 120 S&S 6v Control',
 'vin': None,
 'year': 2021}
2025-12-05 20:01:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788792%27 HTTP/1.1" 200 None
2025-12-05 20:01:28 [ayvens.es] INFO: Saving data for 1788792: {'created_time': 1764964888.585308, 'last_price_update_time': 1764964888.585322, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788792') HTTP/1.1" 204 0
2025-12-05 20:01:28 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/169/21e4f4b71cbd4319954ab2121913ca22/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:28 [ayvens.es] INFO: Found listing with ID: 1788802
2025-12-05 20:01:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788802%27 HTTP/1.1" 200 None
2025-12-05 20:01:28 [ayvens.es] INFO: Scrape type for 1788802: 1
2025-12-05 20:01:28 [ayvens.es] INFO: Extracting data for car ID: 1788802
2025-12-05 20:01:28 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 76b60578-707a-5c68-88aa-3b99f1fffe0e with identifier: 1788802
2025-12-05 20:01:28 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 76b60578-707a-5c68-88aa-3b99f1fffe0e with scrape type 1
2025-12-05 20:01:28 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 76b60578-707a-5c68-88aa-3b99f1fffe0e
2025-12-05 20:01:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F145%252F3017f52e7a764d21855f26e066923e65%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/145/3017f52e7a764d21855f26e066923e65/images/large/1.jpg not downloaded yet
2025-12-05 20:01:28 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/393/2ceccb4749f14a008f6abb24d74704fa/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:28 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:28 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:28 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:28 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0541f10>
2025-12-05 20:01:28 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:28 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:28 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:01:28 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:28 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:28 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:28 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_21edc391-54ca-5273-bb54-7ac17071eea9/21edc391-54ca-5273-bb54-7ac17071eea9_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F540%252F8630530a1353470e91e55b8237bb6df2%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/540/8630530a1353470e91e55b8237bb6df2/images/large/1.jpg saved to cache
2025-12-05 20:01:28 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 21edc391-54ca-5273-bb54-7ac17071eea9
2025-12-05 20:01:28 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 21edc391-54ca-5273-bb54-7ac17071eea9, skipping ID generation
2025-12-05 20:01:28 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 21edc391-54ca-5273-bb54-7ac17071eea9 with scrape type 1
2025-12-05 20:01:28 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 21edc391-54ca-5273-bb54-7ac17071eea9 from tracking dictionaries
2025-12-05 20:01:28 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 21edc391-54ca-5273-bb54-7ac17071eea9 sending to next pipeline
2025-12-05 20:01:29 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 21edc391-54ca-5273-bb54-7ac17071eea9, identifier: 1788794
2025-12-05 20:01:29 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:29 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:29 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:29 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b06de150>
2025-12-05 20:01:29 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:29 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:29 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:01:29 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:29 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:29 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:29 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_680aef6e-7cb6-59d7-aed9-0b805dfe5cd2/680aef6e-7cb6-59d7-aed9-0b805dfe5cd2_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F732%252F1d6760db1a584956b89ef34d22e314f5%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/732/1d6760db1a584956b89ef34d22e314f5/images/large/1.jpg saved to cache
2025-12-05 20:01:29 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 680aef6e-7cb6-59d7-aed9-0b805dfe5cd2
2025-12-05 20:01:29 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 680aef6e-7cb6-59d7-aed9-0b805dfe5cd2, skipping ID generation
2025-12-05 20:01:29 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 680aef6e-7cb6-59d7-aed9-0b805dfe5cd2 with scrape type 1
2025-12-05 20:01:29 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 680aef6e-7cb6-59d7-aed9-0b805dfe5cd2 from tracking dictionaries
2025-12-05 20:01:29 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 680aef6e-7cb6-59d7-aed9-0b805dfe5cd2 sending to next pipeline
2025-12-05 20:01:29 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 680aef6e-7cb6-59d7-aed9-0b805dfe5cd2, identifier: 1788795
2025-12-05 20:01:29 [ayvens.es] INFO: Found listing with ID: 1788803
2025-12-05 20:01:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788803%27 HTTP/1.1" 200 None
2025-12-05 20:01:29 [ayvens.es] INFO: Scrape type for 1788803: 1
2025-12-05 20:01:29 [ayvens.es] INFO: Extracting data for car ID: 1788803
2025-12-05 20:01:29 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 2f6d64c4-4d81-5564-b455-3c13aa56867d with identifier: 1788803
2025-12-05 20:01:29 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 2f6d64c4-4d81-5564-b455-3c13aa56867d with scrape type 1
2025-12-05 20:01:29 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 2f6d64c4-4d81-5564-b455-3c13aa56867d
2025-12-05 20:01:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F438%252Fdc9ba3a35a9b42a0824a9fa23b41f0cc%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/438/dc9ba3a35a9b42a0824a9fa23b41f0cc/images/large/1.jpg not downloaded yet
2025-12-05 20:01:29 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 21edc391-54ca-5273-bb54-7ac17071eea9 with identifier: 1788794
2025-12-05 20:01:29 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 21edc391-54ca-5273-bb54-7ac17071eea9 with identifier: 1788794
2025-12-05 20:01:29 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'renault' containing 'renault' and model 'kangoo express' containing 'express'
2025-12-05 20:01:29 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 21edc391-54ca-5273-bb54-7ac17071eea9 spider: ayvens.es
2025-12-05 20:01:29 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 21edc391-54ca-5273-bb54-7ac17071eea9 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'renault' containing 'renault' and model 'kangoo express' containing 'express'
{'auction_title': '',
 'c_o_2_emission_value': 117,
 'color': 'Pintura',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '21edc391-54ca-5273-bb54-7ac17071eea9',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'KANGOO EXPRESS',
 'odometer': 32041,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_21edc391-54ca-5273-bb54-7ac17071eea9/21edc391-54ca-5273-bb54-7ac17071eea9_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 349590,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/540/8630530a1353470e91e55b8237bb6df2/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-09-18',
 'seats_number': 2,
 'seller_name': None,
 'title': 'RENAULT KANGOO EXPRESS Maxi 2p Blue dCi 70 kW (95CV)',
 'transmission': 'manual',
 'trim': 'Maxi 2p Blue dCi 70 kW (95CV)',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788794%27 HTTP/1.1" 200 None
2025-12-05 20:01:29 [ayvens.es] INFO: Saving data for 1788794: {'created_time': 1764964889.772798, 'last_price_update_time': 1764964889.772809, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788794') HTTP/1.1" 204 0
2025-12-05 20:01:29 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 680aef6e-7cb6-59d7-aed9-0b805dfe5cd2 with identifier: 1788795
2025-12-05 20:01:29 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 680aef6e-7cb6-59d7-aed9-0b805dfe5cd2 with identifier: 1788795
2025-12-05 20:01:30 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'renault' containing 'renault' and model 'kangoo express' containing 'express'
2025-12-05 20:01:30 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 680aef6e-7cb6-59d7-aed9-0b805dfe5cd2 spider: ayvens.es
2025-12-05 20:01:30 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 680aef6e-7cb6-59d7-aed9-0b805dfe5cd2 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'renault' containing 'renault' and model 'kangoo express' containing 'express'
{'auction_title': '',
 'c_o_2_emission_value': 117,
 'color': 'Pintura',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '680aef6e-7cb6-59d7-aed9-0b805dfe5cd2',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'KANGOO EXPRESS',
 'odometer': 32344,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_680aef6e-7cb6-59d7-aed9-0b805dfe5cd2/680aef6e-7cb6-59d7-aed9-0b805dfe5cd2_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 327239,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/732/1d6760db1a584956b89ef34d22e314f5/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-08-20',
 'seats_number': 2,
 'seller_name': None,
 'title': 'RENAULT KANGOO EXPRESS Maxi 2p Blue dCi 70 kW (95CV)',
 'transmission': 'manual',
 'trim': 'Maxi 2p Blue dCi 70 kW (95CV)',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788795%27 HTTP/1.1" 200 None
2025-12-05 20:01:30 [ayvens.es] INFO: Saving data for 1788795: {'created_time': 1764964890.165968, 'last_price_update_time': 1764964890.165981, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788795') HTTP/1.1" 204 0
2025-12-05 20:01:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/177/6d1fb6202489434088daad4478fdc3f2/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:30 [ayvens.es] INFO: Found listing with ID: 1788804
2025-12-05 20:01:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788804%27 HTTP/1.1" 200 None
2025-12-05 20:01:30 [ayvens.es] INFO: Scrape type for 1788804: 1
2025-12-05 20:01:30 [ayvens.es] INFO: Extracting data for car ID: 1788804
2025-12-05 20:01:30 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 9b021b3f-9810-5c0a-b0c2-e633f1b78b26 with identifier: 1788804
2025-12-05 20:01:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9b021b3f-9810-5c0a-b0c2-e633f1b78b26 with scrape type 1
2025-12-05 20:01:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 9b021b3f-9810-5c0a-b0c2-e633f1b78b26
2025-12-05 20:01:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F162%252F1e66f517ca7f490f937e56615fa6fd33%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/162/1e66f517ca7f490f937e56615fa6fd33/images/large/1.jpg not downloaded yet
2025-12-05 20:01:30 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:30 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:30 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:30 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a51c50>
2025-12-05 20:01:30 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:30 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:30 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:01:30 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:30 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:30 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:30 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_fb158477-27d7-589e-acb6-4ea177572fac/fb158477-27d7-589e-acb6-4ea177572fac_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F169%252F21e4f4b71cbd4319954ab2121913ca22%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/169/21e4f4b71cbd4319954ab2121913ca22/images/large/1.jpg saved to cache
2025-12-05 20:01:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item fb158477-27d7-589e-acb6-4ea177572fac
2025-12-05 20:01:30 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: fb158477-27d7-589e-acb6-4ea177572fac, skipping ID generation
2025-12-05 20:01:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item fb158477-27d7-589e-acb6-4ea177572fac with scrape type 1
2025-12-05 20:01:30 [crawlers.middlewares.photo_download_middleware] INFO: Removing item fb158477-27d7-589e-acb6-4ea177572fac from tracking dictionaries
2025-12-05 20:01:30 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item fb158477-27d7-589e-acb6-4ea177572fac sending to next pipeline
2025-12-05 20:01:30 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: fb158477-27d7-589e-acb6-4ea177572fac, identifier: 1788796
2025-12-05 20:01:30 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:30 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:30 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:30 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b054a410>
2025-12-05 20:01:30 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:30 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:30 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1969
2025-12-05 20:01:30 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:30 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:30 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:31 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_964ef805-c965-583b-b118-cc1075504436/964ef805-c965-583b-b118-cc1075504436_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F393%252F2ceccb4749f14a008f6abb24d74704fa%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/393/2ceccb4749f14a008f6abb24d74704fa/images/large/1.jpg saved to cache
2025-12-05 20:01:31 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 964ef805-c965-583b-b118-cc1075504436
2025-12-05 20:01:31 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 964ef805-c965-583b-b118-cc1075504436, skipping ID generation
2025-12-05 20:01:31 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 964ef805-c965-583b-b118-cc1075504436 with scrape type 1
2025-12-05 20:01:31 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 964ef805-c965-583b-b118-cc1075504436 from tracking dictionaries
2025-12-05 20:01:31 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 964ef805-c965-583b-b118-cc1075504436 sending to next pipeline
2025-12-05 20:01:31 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 964ef805-c965-583b-b118-cc1075504436, identifier: 1788797
2025-12-05 20:01:31 [ayvens.es] INFO: Found listing with ID: 1788805
2025-12-05 20:01:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788805%27 HTTP/1.1" 200 None
2025-12-05 20:01:31 [ayvens.es] INFO: Scrape type for 1788805: 1
2025-12-05 20:01:31 [ayvens.es] INFO: Extracting data for car ID: 1788805
2025-12-05 20:01:31 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 2eda0255-6c15-52df-a582-06a3e26cd85b with identifier: 1788805
2025-12-05 20:01:31 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 2eda0255-6c15-52df-a582-06a3e26cd85b with scrape type 1
2025-12-05 20:01:31 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 2eda0255-6c15-52df-a582-06a3e26cd85b
2025-12-05 20:01:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F586%252Ffc53da839c5b4cd89d25318843373e53%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/586/fc53da839c5b4cd89d25318843373e53/images/large/1.jpg not downloaded yet
2025-12-05 20:01:31 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: fb158477-27d7-589e-acb6-4ea177572fac with identifier: 1788796
2025-12-05 20:01:31 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:01:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:01:31 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: fb158477-27d7-589e-acb6-4ea177572fac with identifier: 1788796
2025-12-05 20:01:31 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'renault' containing 'renault' and model 'master' containing 'master'
2025-12-05 20:01:31 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: fb158477-27d7-589e-acb6-4ea177572fac spider: ayvens.es
2025-12-05 20:01:31 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: fb158477-27d7-589e-acb6-4ea177572fac due to rule: makes_models_not_allowed.json, error: Rejected due to make 'renault' containing 'renault' and model 'master' containing 'master'
{'auction_title': '',
 'c_o_2_emission_value': 191,
 'color': 'Amarillo',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '2.3L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'fb158477-27d7-589e-acb6-4ea177572fac',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'MASTER',
 'odometer': 44228,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_fb158477-27d7-589e-acb6-4ea177572fac/fb158477-27d7-589e-acb6-4ea177572fac_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 361274,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/169/21e4f4b71cbd4319954ab2121913ca22/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-11-25',
 'seats_number': 3,
 'seller_name': None,
 'title': 'RENAULT MASTER Furgon T L3H2 3500 Bl dCi 100 kW(135CV)',
 'transmission': 'manual',
 'trim': 'Furgon T L3H2 3500 Bl dCi 100 kW(135CV)',
 'vin': None,
 'year': 2019}
2025-12-05 20:01:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788796%27 HTTP/1.1" 200 None
2025-12-05 20:01:31 [ayvens.es] INFO: Saving data for 1788796: {'created_time': 1764964891.849721, 'last_price_update_time': 1764964891.849732, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788796') HTTP/1.1" 204 0
2025-12-05 20:01:31 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 964ef805-c965-583b-b118-cc1075504436 with identifier: 1788797
2025-12-05 20:01:31 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 964ef805-c965-583b-b118-cc1075504436 with identifier: 1788797
2025-12-05 20:01:31 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 964ef805-c965-583b-b118-cc1075504436 with identifier: 1788797 to the API
2025-12-05 20:01:31 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:32 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:32 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/393/2ceccb4749f14a008f6abb24d74704fa/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 116,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '964ef805-c965-583b-b118-cc1075504436',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': '2008',
 'odometer': 118199,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_964ef805-c965-583b-b118-cc1075504436/964ef805-c965-583b-b118-cc1075504436_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 345625,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/393/2ceccb4749f14a008f6abb24d74704fa/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-06-17',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT 2008 Active Pack BlueHDI 81kW (110CV)',
 'transmission': 'manual',
 'trim': 'Active Pack BlueHDI 81kW (110CV)',
 'vin': None,
 'year': 2021}
2025-12-05 20:01:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788797%27 HTTP/1.1" 200 None
2025-12-05 20:01:32 [ayvens.es] INFO: Saving data for 1788797: {'created_time': 1764964892.1825, 'last_price_update_time': 1764964892.182512, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788797') HTTP/1.1" 204 0
2025-12-05 20:01:32 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/548/6d38b248165d47e5b8da31b676a49adf/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:32 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/986/4d703f59b2f7414ca01323bcd2d37a7f/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:32 [ayvens.es] INFO: Found listing with ID: 1788806
2025-12-05 20:01:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788806%27 HTTP/1.1" 200 None
2025-12-05 20:01:32 [ayvens.es] INFO: Scrape type for 1788806: 1
2025-12-05 20:01:32 [ayvens.es] INFO: Extracting data for car ID: 1788806
2025-12-05 20:01:32 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 561288b8-a2c7-5903-9453-25d19d8728ac with identifier: 1788806
2025-12-05 20:01:32 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 561288b8-a2c7-5903-9453-25d19d8728ac with scrape type 1
2025-12-05 20:01:32 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 561288b8-a2c7-5903-9453-25d19d8728ac
2025-12-05 20:01:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F234%252Fd6a5c04fd82f4b0fbd670e2f8d3e134b%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/234/d6a5c04fd82f4b0fbd670e2f8d3e134b/images/large/1.jpg not downloaded yet
2025-12-05 20:01:32 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:32 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:32 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:32 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b05f6050>
2025-12-05 20:01:32 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:32 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:32 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:32 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:32 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:32 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:32 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_ee87e13e-1ec9-5552-b595-16f2dcd8af4d/ee87e13e-1ec9-5552-b595-16f2dcd8af4d_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F177%252F6d1fb6202489434088daad4478fdc3f2%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/177/6d1fb6202489434088daad4478fdc3f2/images/large/1.jpg saved to cache
2025-12-05 20:01:32 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item ee87e13e-1ec9-5552-b595-16f2dcd8af4d
2025-12-05 20:01:32 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: ee87e13e-1ec9-5552-b595-16f2dcd8af4d, skipping ID generation
2025-12-05 20:01:32 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ee87e13e-1ec9-5552-b595-16f2dcd8af4d with scrape type 1
2025-12-05 20:01:32 [crawlers.middlewares.photo_download_middleware] INFO: Removing item ee87e13e-1ec9-5552-b595-16f2dcd8af4d from tracking dictionaries
2025-12-05 20:01:32 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item ee87e13e-1ec9-5552-b595-16f2dcd8af4d sending to next pipeline
2025-12-05 20:01:32 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: ee87e13e-1ec9-5552-b595-16f2dcd8af4d, identifier: 1788798
2025-12-05 20:01:32 [ayvens.es] INFO: Found listing with ID: 1788807
2025-12-05 20:01:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788807%27 HTTP/1.1" 200 None
2025-12-05 20:01:32 [ayvens.es] INFO: Scrape type for 1788807: 1
2025-12-05 20:01:32 [ayvens.es] INFO: Extracting data for car ID: 1788807
2025-12-05 20:01:32 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 38b36fbb-cc49-5c85-9667-a272c908c0ef with identifier: 1788807
2025-12-05 20:01:32 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 38b36fbb-cc49-5c85-9667-a272c908c0ef with scrape type 1
2025-12-05 20:01:32 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 38b36fbb-cc49-5c85-9667-a272c908c0ef
2025-12-05 20:01:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F366%252F6c8ca8a3fdf94f3eaa113a62ed57c77a%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/366/6c8ca8a3fdf94f3eaa113a62ed57c77a/images/large/1.jpg not downloaded yet
2025-12-05 20:01:32 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: ee87e13e-1ec9-5552-b595-16f2dcd8af4d with identifier: 1788798
2025-12-05 20:01:32 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: ee87e13e-1ec9-5552-b595-16f2dcd8af4d with identifier: 1788798
2025-12-05 20:01:32 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'peugeot', 'model': '2008', 'trim': 'active puretech 100 s&s bvm6', 'title': 'peugeot 2008 active puretech 100 s&s bvm6', 'price': 0, 'odometer': 82420, 'odometer_reading_unit': 'km', 'registration_date': '2020-10-26', 'year': 2020, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.2l', 'c_o_2_emission_value': 122, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'ee87e13e-1ec9-5552-b595-16f2dcd8af4d', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ee87e13e-1ec9-5552-b595-16f2dcd8af4d/ee87e13e-1ec9-5552-b595-16f2dcd8af4d_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/177/6d1fb6202489434088daad4478fdc3f2/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 333677}], 'current_year_diff': 5, 'age_in_months': 61, 'make_model': 'peugeot 2008', 'sanitized_make': 'peugeot', 'odometer_in_km': 82420}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 61,
     'auction_title': '',
     'c_o_2_emission_value': 122,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.2l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': 'ee87e13e-1ec9-5552-b595-16f2dcd8af4d',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'peugeot',
     'make_model': 'peugeot 2008',
     'model': '2008',
     'odometer': 82420,
     'odometer_in_km': 82420,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ee87e13e-1ec9-5552-b595-16f2dcd8af4d/ee87e13e-1ec9-5552-b595-16f2dcd8af4d_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 333677,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/177/6d1fb6202489434088daad4478fdc3f2/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-10-26',
     'sanitized_make': 'peugeot',
     'seats_number': 5,
     'seller_name': None,
     'title': 'peugeot 2008 active puretech 100 s&s bvm6',
     'transmission': 'manual',
     'trim': 'active puretech 100 s&s bvm6',
     'vin': None,
     'year': 2020}
2025-12-05 20:01:32 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: PURETECH_FOR_COUNTRY failed for item: ee87e13e-1ec9-5552-b595-16f2dcd8af4d spider: ayvens.es
2025-12-05 20:01:33 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: ee87e13e-1ec9-5552-b595-16f2dcd8af4d due to rule: puretech_for_country.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'peugeot', 'model': '2008', 'trim': 'active puretech 100 s&s bvm6', 'title': 'peugeot 2008 active puretech 100 s&s bvm6', 'price': 0, 'odometer': 82420, 'odometer_reading_unit': 'km', 'registration_date': '2020-10-26', 'year': 2020, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.2l', 'c_o_2_emission_value': 122, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'ee87e13e-1ec9-5552-b595-16f2dcd8af4d', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ee87e13e-1ec9-5552-b595-16f2dcd8af4d/ee87e13e-1ec9-5552-b595-16f2dcd8af4d_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/177/6d1fb6202489434088daad4478fdc3f2/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 333677}], 'current_year_diff': 5, 'age_in_months': 61, 'make_model': 'peugeot 2008', 'sanitized_make': 'peugeot', 'odometer_in_km': 82420}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 61,
     'auction_title': '',
     'c_o_2_emission_value': 122,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.2l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': 'ee87e13e-1ec9-5552-b595-16f2dcd8af4d',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'peugeot',
     'make_model': 'peugeot 2008',
     'model': '2008',
     'odometer': 82420,
     'odometer_in_km': 82420,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ee87e13e-1ec9-5552-b595-16f2dcd8af4d/ee87e13e-1ec9-5552-b595-16f2dcd8af4d_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 333677,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/177/6d1fb6202489434088daad4478fdc3f2/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-10-26',
     'sanitized_make': 'peugeot',
     'seats_number': 5,
     'seller_name': None,
     'title': 'peugeot 2008 active puretech 100 s&s bvm6',
     'transmission': 'manual',
     'trim': 'active puretech 100 s&s bvm6',
     'vin': None,
     'year': 2020}
{'auction_title': '',
 'c_o_2_emission_value': 122,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.2L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': 'ee87e13e-1ec9-5552-b595-16f2dcd8af4d',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': '2008',
 'odometer': 82420,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ee87e13e-1ec9-5552-b595-16f2dcd8af4d/ee87e13e-1ec9-5552-b595-16f2dcd8af4d_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 333677,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/177/6d1fb6202489434088daad4478fdc3f2/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-10-26',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT 2008 Active Puretech 100 S&S BVM6',
 'transmission': 'manual',
 'trim': 'Active Puretech 100 S&S BVM6',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788798%27 HTTP/1.1" 200 None
2025-12-05 20:01:33 [ayvens.es] INFO: Saving data for 1788798: {'created_time': 1764964893.054762, 'last_price_update_time': 1764964893.054774, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788798') HTTP/1.1" 204 0
2025-12-05 20:01:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/145/3017f52e7a764d21855f26e066923e65/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/358/be65dc5152fe4da0891f4abb76e51d27/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:33 [ayvens.es] INFO: Found listing with ID: 1788808
2025-12-05 20:01:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788808%27 HTTP/1.1" 200 None
2025-12-05 20:01:33 [ayvens.es] INFO: Scrape type for 1788808: 1
2025-12-05 20:01:33 [ayvens.es] INFO: Extracting data for car ID: 1788808
2025-12-05 20:01:33 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 35150818-ecf9-5843-840c-0b67f3c2ac3c with identifier: 1788808
2025-12-05 20:01:33 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 35150818-ecf9-5843-840c-0b67f3c2ac3c with scrape type 1
2025-12-05 20:01:33 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 35150818-ecf9-5843-840c-0b67f3c2ac3c
2025-12-05 20:01:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F382%252Fc51d6af8a84646b1a411004c463671f3%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/382/c51d6af8a84646b1a411004c463671f3/images/large/1.jpg not downloaded yet
2025-12-05 20:01:33 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:33 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:33 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:33 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b2a56790>
2025-12-05 20:01:33 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:33 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:33 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:01:33 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:33 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:33 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:33 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_8aec3a22-a764-54c5-9a7a-8c0f3730c908/8aec3a22-a764-54c5-9a7a-8c0f3730c908_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F548%252F6d38b248165d47e5b8da31b676a49adf%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/548/6d38b248165d47e5b8da31b676a49adf/images/large/1.jpg saved to cache
2025-12-05 20:01:33 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 8aec3a22-a764-54c5-9a7a-8c0f3730c908
2025-12-05 20:01:33 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 8aec3a22-a764-54c5-9a7a-8c0f3730c908, skipping ID generation
2025-12-05 20:01:33 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 8aec3a22-a764-54c5-9a7a-8c0f3730c908 with scrape type 1
2025-12-05 20:01:33 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 8aec3a22-a764-54c5-9a7a-8c0f3730c908 from tracking dictionaries
2025-12-05 20:01:33 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 8aec3a22-a764-54c5-9a7a-8c0f3730c908 sending to next pipeline
2025-12-05 20:01:33 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 8aec3a22-a764-54c5-9a7a-8c0f3730c908, identifier: 1788799
2025-12-05 20:01:33 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:33 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:33 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:33 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0cd6f10>
2025-12-05 20:01:33 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:33 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:33 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:01:33 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:33 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:33 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_b9670650-994c-576f-bb79-e8dff1806856/b9670650-994c-576f-bb79-e8dff1806856_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F986%252F4d703f59b2f7414ca01323bcd2d37a7f%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/986/4d703f59b2f7414ca01323bcd2d37a7f/images/large/1.jpg saved to cache
2025-12-05 20:01:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item b9670650-994c-576f-bb79-e8dff1806856
2025-12-05 20:01:34 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: b9670650-994c-576f-bb79-e8dff1806856, skipping ID generation
2025-12-05 20:01:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b9670650-994c-576f-bb79-e8dff1806856 with scrape type 1
2025-12-05 20:01:34 [crawlers.middlewares.photo_download_middleware] INFO: Removing item b9670650-994c-576f-bb79-e8dff1806856 from tracking dictionaries
2025-12-05 20:01:34 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item b9670650-994c-576f-bb79-e8dff1806856 sending to next pipeline
2025-12-05 20:01:34 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: b9670650-994c-576f-bb79-e8dff1806856, identifier: 1788800
2025-12-05 20:01:34 [ayvens.es] INFO: Found listing with ID: 1788809
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788809%27 HTTP/1.1" 200 None
2025-12-05 20:01:34 [ayvens.es] INFO: Scrape type for 1788809: 1
2025-12-05 20:01:34 [ayvens.es] INFO: Extracting data for car ID: 1788809
2025-12-05 20:01:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 5798ae1e-5084-5fe0-929e-173309c1b413 with identifier: 1788809
2025-12-05 20:01:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 5798ae1e-5084-5fe0-929e-173309c1b413 with scrape type 1
2025-12-05 20:01:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 5798ae1e-5084-5fe0-929e-173309c1b413
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F621%252F1bb5fa03ea864089b727092b766cf197%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/621/1bb5fa03ea864089b727092b766cf197/images/large/1.jpg not downloaded yet
2025-12-05 20:01:34 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 8aec3a22-a764-54c5-9a7a-8c0f3730c908 with identifier: 1788799
2025-12-05 20:01:34 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 8aec3a22-a764-54c5-9a7a-8c0f3730c908 with identifier: 1788799
2025-12-05 20:01:34 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 8aec3a22-a764-54c5-9a7a-8c0f3730c908 with identifier: 1788799 to the API
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:34 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/548/6d38b248165d47e5b8da31b676a49adf/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 115,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 3,
 'engine': '1.2L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '8aec3a22-a764-54c5-9a7a-8c0f3730c908',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'FIAT',
 'model': '500',
 'odometer': 104368,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_8aec3a22-a764-54c5-9a7a-8c0f3730c908/8aec3a22-a764-54c5-9a7a-8c0f3730c908_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 308526,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/548/6d38b248165d47e5b8da31b676a49adf/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2017-08-31',
 'seats_number': 4,
 'seller_name': None,
 'title': 'FIAT 500 1.2 8v 51kW (69CV) Lounge',
 'transmission': 'manual',
 'trim': '1.2 8v 51kW (69CV) Lounge',
 'vin': None,
 'year': 2017}
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788799%27 HTTP/1.1" 200 None
2025-12-05 20:01:34 [ayvens.es] INFO: Saving data for 1788799: {'created_time': 1764964894.511775, 'last_price_update_time': 1764964894.511784, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788799') HTTP/1.1" 204 0
2025-12-05 20:01:34 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: b9670650-994c-576f-bb79-e8dff1806856 with identifier: 1788800
2025-12-05 20:01:34 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: b9670650-994c-576f-bb79-e8dff1806856 with identifier: 1788800
2025-12-05 20:01:34 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: b9670650-994c-576f-bb79-e8dff1806856 with identifier: 1788800 to the API
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:34 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/986/4d703f59b2f7414ca01323bcd2d37a7f/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 159,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.6L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': 'b9670650-994c-576f-bb79-e8dff1806856',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'HYUNDAI',
 'model': 'TUCSON',
 'odometer': 111816,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_b9670650-994c-576f-bb79-e8dff1806856/b9670650-994c-576f-bb79-e8dff1806856_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 365053,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/986/4d703f59b2f7414ca01323bcd2d37a7f/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-05-22',
 'seats_number': 5,
 'seller_name': None,
 'title': 'HYUNDAI TUCSON 1.6 T-GDi 130 kW Premium DCT-7',
 'transmission': 'manual',
 'trim': '1.6 T-GDi 130 kW Premium DCT-7',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788800%27 HTTP/1.1" 200 None
2025-12-05 20:01:34 [ayvens.es] INFO: Saving data for 1788800: {'created_time': 1764964894.908748, 'last_price_update_time': 1764964894.908757, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788800') HTTP/1.1" 204 0
2025-12-05 20:01:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/162/1e66f517ca7f490f937e56615fa6fd33/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/438/dc9ba3a35a9b42a0824a9fa23b41f0cc/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:34 [ayvens.es] INFO: Found listing with ID: 1788810
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788810%27 HTTP/1.1" 200 None
2025-12-05 20:01:34 [ayvens.es] INFO: Scrape type for 1788810: 1
2025-12-05 20:01:34 [ayvens.es] INFO: Extracting data for car ID: 1788810
2025-12-05 20:01:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 6af472fb-bac7-5b7b-b8fb-5e2917a08a62 with identifier: 1788810
2025-12-05 20:01:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6af472fb-bac7-5b7b-b8fb-5e2917a08a62 with scrape type 1
2025-12-05 20:01:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 6af472fb-bac7-5b7b-b8fb-5e2917a08a62
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F382%252Ffd90fbecf1634afeb1c8ed4da32fc7de%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/382/fd90fbecf1634afeb1c8ed4da32fc7de/images/large/1.jpg not downloaded yet
2025-12-05 20:01:34 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:34 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:34 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:34 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0451bd0>
2025-12-05 20:01:34 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:34 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:35 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:01:35 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:35 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:35 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:35 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_76b60578-707a-5c68-88aa-3b99f1fffe0e/76b60578-707a-5c68-88aa-3b99f1fffe0e_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F145%252F3017f52e7a764d21855f26e066923e65%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/145/3017f52e7a764d21855f26e066923e65/images/large/1.jpg saved to cache
2025-12-05 20:01:35 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 76b60578-707a-5c68-88aa-3b99f1fffe0e
2025-12-05 20:01:35 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 76b60578-707a-5c68-88aa-3b99f1fffe0e, skipping ID generation
2025-12-05 20:01:35 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 76b60578-707a-5c68-88aa-3b99f1fffe0e with scrape type 1
2025-12-05 20:01:35 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 76b60578-707a-5c68-88aa-3b99f1fffe0e from tracking dictionaries
2025-12-05 20:01:35 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 76b60578-707a-5c68-88aa-3b99f1fffe0e sending to next pipeline
2025-12-05 20:01:35 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 76b60578-707a-5c68-88aa-3b99f1fffe0e, identifier: 1788802
2025-12-05 20:01:35 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:35 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:35 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:35 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b047bbd0>
2025-12-05 20:01:35 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:35 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:35 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:35 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:35 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:35 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:35 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_a3b68e8b-8606-5ed0-8e32-07be20d1db5c/a3b68e8b-8606-5ed0-8e32-07be20d1db5c_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F358%252Fbe65dc5152fe4da0891f4abb76e51d27%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/358/be65dc5152fe4da0891f4abb76e51d27/images/large/1.jpg saved to cache
2025-12-05 20:01:35 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item a3b68e8b-8606-5ed0-8e32-07be20d1db5c
2025-12-05 20:01:35 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: a3b68e8b-8606-5ed0-8e32-07be20d1db5c, skipping ID generation
2025-12-05 20:01:35 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a3b68e8b-8606-5ed0-8e32-07be20d1db5c with scrape type 1
2025-12-05 20:01:35 [crawlers.middlewares.photo_download_middleware] INFO: Removing item a3b68e8b-8606-5ed0-8e32-07be20d1db5c from tracking dictionaries
2025-12-05 20:01:35 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item a3b68e8b-8606-5ed0-8e32-07be20d1db5c sending to next pipeline
2025-12-05 20:01:35 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: a3b68e8b-8606-5ed0-8e32-07be20d1db5c, identifier: 1788801
2025-12-05 20:01:35 [ayvens.es] INFO: Found listing with ID: 1788811
2025-12-05 20:01:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788811%27 HTTP/1.1" 200 None
2025-12-05 20:01:35 [ayvens.es] INFO: Scrape type for 1788811: 1
2025-12-05 20:01:35 [ayvens.es] INFO: Extracting data for car ID: 1788811
2025-12-05 20:01:35 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 7ba0cbb9-0036-5f93-af73-83b770053d0c with identifier: 1788811
2025-12-05 20:01:35 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 7ba0cbb9-0036-5f93-af73-83b770053d0c with scrape type 1
2025-12-05 20:01:35 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 7ba0cbb9-0036-5f93-af73-83b770053d0c
2025-12-05 20:01:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F455%252Fd47c0ac527294bdeae6e23031d0625de%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/455/d47c0ac527294bdeae6e23031d0625de/images/large/1.jpg not downloaded yet
2025-12-05 20:01:35 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 76b60578-707a-5c68-88aa-3b99f1fffe0e with identifier: 1788802
2025-12-05 20:01:35 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:01:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:01:35 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 76b60578-707a-5c68-88aa-3b99f1fffe0e with identifier: 1788802
2025-12-05 20:01:35 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 76b60578-707a-5c68-88aa-3b99f1fffe0e with identifier: 1788802 to the API
2025-12-05 20:01:35 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:36 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:36 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/145/3017f52e7a764d21855f26e066923e65/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 124,
 'color': 'Verde',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '76b60578-707a-5c68-88aa-3b99f1fffe0e',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'MINI',
 'model': 'COUNTRYMAN',
 'odometer': 43622,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_76b60578-707a-5c68-88aa-3b99f1fffe0e/76b60578-707a-5c68-88aa-3b99f1fffe0e_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 382589,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/145/3017f52e7a764d21855f26e066923e65/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-09-20',
 'seats_number': 5,
 'seller_name': None,
 'title': 'MINI COUNTRYMAN Cooper D',
 'transmission': 'manual',
 'trim': 'Cooper D',
 'vin': None,
 'year': 2021}
2025-12-05 20:01:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788802%27 HTTP/1.1" 200 None
2025-12-05 20:01:36 [ayvens.es] INFO: Saving data for 1788802: {'created_time': 1764964896.188624, 'last_price_update_time': 1764964896.188637, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788802') HTTP/1.1" 204 0
2025-12-05 20:01:36 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: a3b68e8b-8606-5ed0-8e32-07be20d1db5c with identifier: 1788801
2025-12-05 20:01:36 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: a3b68e8b-8606-5ed0-8e32-07be20d1db5c with identifier: 1788801
2025-12-05 20:01:36 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'renault' containing 'renault' and model 'kangoo' containing 'kangoo'
2025-12-05 20:01:36 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: a3b68e8b-8606-5ed0-8e32-07be20d1db5c spider: ayvens.es
2025-12-05 20:01:36 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: a3b68e8b-8606-5ed0-8e32-07be20d1db5c due to rule: makes_models_not_allowed.json, error: Rejected due to make 'renault' containing 'renault' and model 'kangoo' containing 'kangoo'
{'auction_title': '',
 'c_o_2_emission_value': 123,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'a3b68e8b-8606-5ed0-8e32-07be20d1db5c',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'KANGOO',
 'odometer': 124984,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_a3b68e8b-8606-5ed0-8e32-07be20d1db5c/a3b68e8b-8606-5ed0-8e32-07be20d1db5c_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 345083,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/358/be65dc5152fe4da0891f4abb76e51d27/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-04-25',
 'seats_number': 5,
 'seller_name': None,
 'title': 'RENAULT KANGOO Profesional N1 Energy dCi 66kW (90CV)',
 'transmission': 'manual',
 'trim': 'Profesional N1 Energy dCi 66kW (90CV)',
 'vin': None,
 'year': 2019}
2025-12-05 20:01:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788801%27 HTTP/1.1" 200 None
2025-12-05 20:01:36 [ayvens.es] INFO: Saving data for 1788801: {'created_time': 1764964896.42717, 'last_price_update_time': 1764964896.427179, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788801') HTTP/1.1" 204 0
2025-12-05 20:01:36 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/586/fc53da839c5b4cd89d25318843373e53/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:36 [ayvens.es] INFO: Found listing with ID: 1788812
2025-12-05 20:01:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788812%27 HTTP/1.1" 200 None
2025-12-05 20:01:36 [ayvens.es] INFO: Scrape type for 1788812: 1
2025-12-05 20:01:36 [ayvens.es] INFO: Extracting data for car ID: 1788812
2025-12-05 20:01:36 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 6492ac7c-2210-5950-b04d-43b96b7f4cbf with identifier: 1788812
2025-12-05 20:01:36 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6492ac7c-2210-5950-b04d-43b96b7f4cbf with scrape type 1
2025-12-05 20:01:36 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 6492ac7c-2210-5950-b04d-43b96b7f4cbf
2025-12-05 20:01:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F640%252F4d70e3c1bd1f447098a863661f2fd6bc%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:36 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/640/4d70e3c1bd1f447098a863661f2fd6bc/images/large/1.jpg not downloaded yet
2025-12-05 20:01:36 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:36 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:36 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:36 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b05fae90>
2025-12-05 20:01:36 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:36 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:36 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1969
2025-12-05 20:01:36 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:36 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:36 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:36 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_9b021b3f-9810-5c0a-b0c2-e633f1b78b26/9b021b3f-9810-5c0a-b0c2-e633f1b78b26_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F162%252F1e66f517ca7f490f937e56615fa6fd33%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:36 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/162/1e66f517ca7f490f937e56615fa6fd33/images/large/1.jpg saved to cache
2025-12-05 20:01:36 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 9b021b3f-9810-5c0a-b0c2-e633f1b78b26
2025-12-05 20:01:36 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 9b021b3f-9810-5c0a-b0c2-e633f1b78b26, skipping ID generation
2025-12-05 20:01:36 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9b021b3f-9810-5c0a-b0c2-e633f1b78b26 with scrape type 1
2025-12-05 20:01:36 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 9b021b3f-9810-5c0a-b0c2-e633f1b78b26 from tracking dictionaries
2025-12-05 20:01:36 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 9b021b3f-9810-5c0a-b0c2-e633f1b78b26 sending to next pipeline
2025-12-05 20:01:36 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 9b021b3f-9810-5c0a-b0c2-e633f1b78b26, identifier: 1788804
2025-12-05 20:01:36 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:36 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:36 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:36 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0bd5ad0>
2025-12-05 20:01:36 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:36 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:37 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:01:37 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:37 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:37 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:37 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_2f6d64c4-4d81-5564-b455-3c13aa56867d/2f6d64c4-4d81-5564-b455-3c13aa56867d_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F438%252Fdc9ba3a35a9b42a0824a9fa23b41f0cc%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:37 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/438/dc9ba3a35a9b42a0824a9fa23b41f0cc/images/large/1.jpg saved to cache
2025-12-05 20:01:37 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 2f6d64c4-4d81-5564-b455-3c13aa56867d
2025-12-05 20:01:37 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 2f6d64c4-4d81-5564-b455-3c13aa56867d, skipping ID generation
2025-12-05 20:01:37 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 2f6d64c4-4d81-5564-b455-3c13aa56867d with scrape type 1
2025-12-05 20:01:37 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 2f6d64c4-4d81-5564-b455-3c13aa56867d from tracking dictionaries
2025-12-05 20:01:37 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 2f6d64c4-4d81-5564-b455-3c13aa56867d sending to next pipeline
2025-12-05 20:01:37 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 2f6d64c4-4d81-5564-b455-3c13aa56867d, identifier: 1788803
2025-12-05 20:01:37 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/234/d6a5c04fd82f4b0fbd670e2f8d3e134b/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:37 [ayvens.es] INFO: Found listing with ID: 1788813
2025-12-05 20:01:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788813%27 HTTP/1.1" 200 None
2025-12-05 20:01:37 [ayvens.es] INFO: Scrape type for 1788813: 1
2025-12-05 20:01:37 [ayvens.es] INFO: Extracting data for car ID: 1788813
2025-12-05 20:01:37 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 2fe30fc3-4c86-5e5a-9db0-652dec8cc6b6 with identifier: 1788813
2025-12-05 20:01:37 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 2fe30fc3-4c86-5e5a-9db0-652dec8cc6b6 with scrape type 1
2025-12-05 20:01:37 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 2fe30fc3-4c86-5e5a-9db0-652dec8cc6b6
2025-12-05 20:01:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F903%252Fb4e1f4cd501b472e91bcfcb1c6ea17ac%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:37 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/903/b4e1f4cd501b472e91bcfcb1c6ea17ac/images/large/1.jpg not downloaded yet
2025-12-05 20:01:37 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 9b021b3f-9810-5c0a-b0c2-e633f1b78b26 with identifier: 1788804
2025-12-05 20:01:37 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 9b021b3f-9810-5c0a-b0c2-e633f1b78b26 with identifier: 1788804
2025-12-05 20:01:37 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 9b021b3f-9810-5c0a-b0c2-e633f1b78b26 with identifier: 1788804 to the API
2025-12-05 20:01:37 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:37 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:37 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/162/1e66f517ca7f490f937e56615fa6fd33/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 147,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '9b021b3f-9810-5c0a-b0c2-e633f1b78b26',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'AUDI',
 'model': 'Q5',
 'odometer': 77355,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_9b021b3f-9810-5c0a-b0c2-e633f1b78b26/9b021b3f-9810-5c0a-b0c2-e633f1b78b26_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 360096,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/162/1e66f517ca7f490f937e56615fa6fd33/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2022-06-30',
 'seats_number': 5,
 'seller_name': None,
 'title': 'AUDI Q5 35 TDI 120kW (163CV) S tronic',
 'transmission': 'manual',
 'trim': '35 TDI 120kW (163CV) S tronic',
 'vin': None,
 'year': 2022}
2025-12-05 20:01:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788804%27 HTTP/1.1" 200 None
2025-12-05 20:01:37 [ayvens.es] INFO: Saving data for 1788804: {'created_time': 1764964897.834786, 'last_price_update_time': 1764964897.834798, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788804') HTTP/1.1" 204 0
2025-12-05 20:01:37 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 2f6d64c4-4d81-5564-b455-3c13aa56867d with identifier: 1788803
2025-12-05 20:01:37 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 2f6d64c4-4d81-5564-b455-3c13aa56867d with identifier: 1788803
2025-12-05 20:01:37 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 2f6d64c4-4d81-5564-b455-3c13aa56867d with identifier: 1788803 to the API
2025-12-05 20:01:37 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:38 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:38 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/438/dc9ba3a35a9b42a0824a9fa23b41f0cc/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 137,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '2f6d64c4-4d81-5564-b455-3c13aa56867d',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'DS AUTOMOBILES',
 'model': 'DS 7 CROSSBACK',
 'odometer': 135821,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_2f6d64c4-4d81-5564-b455-3c13aa56867d/2f6d64c4-4d81-5564-b455-3c13aa56867d_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 351939,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/438/dc9ba3a35a9b42a0824a9fa23b41f0cc/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-11-10',
 'seats_number': 5,
 'seller_name': None,
 'title': 'DS AUTOMOBILES DS 7 CROSSBACK BlueHDi DE 96kW (130CV) AT Bastille +',
 'transmission': 'manual',
 'trim': 'BlueHDi DE 96kW (130CV) AT Bastille +',
 'vin': None,
 'year': 2021}
2025-12-05 20:01:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788803%27 HTTP/1.1" 200 None
2025-12-05 20:01:38 [ayvens.es] INFO: Saving data for 1788803: {'created_time': 1764964898.239641, 'last_price_update_time': 1764964898.23965, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788803') HTTP/1.1" 204 0
2025-12-05 20:01:38 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/366/6c8ca8a3fdf94f3eaa113a62ed57c77a/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:38 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/382/c51d6af8a84646b1a411004c463671f3/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:38 [ayvens.es] INFO: Found listing with ID: 1788814
2025-12-05 20:01:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788814%27 HTTP/1.1" 200 None
2025-12-05 20:01:38 [ayvens.es] INFO: Scrape type for 1788814: 1
2025-12-05 20:01:38 [ayvens.es] INFO: Extracting data for car ID: 1788814
2025-12-05 20:01:38 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 1d5606eb-fcc8-5c67-ad8c-4f32d53905eb with identifier: 1788814
2025-12-05 20:01:38 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 1d5606eb-fcc8-5c67-ad8c-4f32d53905eb with scrape type 1
2025-12-05 20:01:38 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 1d5606eb-fcc8-5c67-ad8c-4f32d53905eb
2025-12-05 20:01:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F530%252F4bd8594581704e06a0412301438ec582%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:38 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/530/4bd8594581704e06a0412301438ec582/images/large/1.jpg not downloaded yet
2025-12-05 20:01:38 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:38 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:38 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:38 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a439d0>
2025-12-05 20:01:38 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:38 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:38 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:01:38 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:38 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:38 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:38 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_2eda0255-6c15-52df-a582-06a3e26cd85b/2eda0255-6c15-52df-a582-06a3e26cd85b_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F586%252Ffc53da839c5b4cd89d25318843373e53%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:38 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/586/fc53da839c5b4cd89d25318843373e53/images/large/1.jpg saved to cache
2025-12-05 20:01:38 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 2eda0255-6c15-52df-a582-06a3e26cd85b
2025-12-05 20:01:38 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 2eda0255-6c15-52df-a582-06a3e26cd85b, skipping ID generation
2025-12-05 20:01:38 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 2eda0255-6c15-52df-a582-06a3e26cd85b with scrape type 1
2025-12-05 20:01:38 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 2eda0255-6c15-52df-a582-06a3e26cd85b from tracking dictionaries
2025-12-05 20:01:38 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 2eda0255-6c15-52df-a582-06a3e26cd85b sending to next pipeline
2025-12-05 20:01:38 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 2eda0255-6c15-52df-a582-06a3e26cd85b, identifier: 1788805
2025-12-05 20:01:38 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:38 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:38 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:38 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b04413d0>
2025-12-05 20:01:38 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:38 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:38 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:01:38 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:39 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:39 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:39 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_561288b8-a2c7-5903-9453-25d19d8728ac/561288b8-a2c7-5903-9453-25d19d8728ac_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F234%252Fd6a5c04fd82f4b0fbd670e2f8d3e134b%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:39 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/234/d6a5c04fd82f4b0fbd670e2f8d3e134b/images/large/1.jpg saved to cache
2025-12-05 20:01:39 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 561288b8-a2c7-5903-9453-25d19d8728ac
2025-12-05 20:01:39 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 561288b8-a2c7-5903-9453-25d19d8728ac, skipping ID generation
2025-12-05 20:01:39 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 561288b8-a2c7-5903-9453-25d19d8728ac with scrape type 1
2025-12-05 20:01:39 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 561288b8-a2c7-5903-9453-25d19d8728ac from tracking dictionaries
2025-12-05 20:01:39 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 561288b8-a2c7-5903-9453-25d19d8728ac sending to next pipeline
2025-12-05 20:01:39 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 561288b8-a2c7-5903-9453-25d19d8728ac, identifier: 1788806
2025-12-05 20:01:39 [ayvens.es] INFO: Found listing with ID: 1788815
2025-12-05 20:01:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788815%27 HTTP/1.1" 200 None
2025-12-05 20:01:39 [ayvens.es] INFO: Scrape type for 1788815: 1
2025-12-05 20:01:39 [ayvens.es] INFO: Extracting data for car ID: 1788815
2025-12-05 20:01:39 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: a8ade30f-b872-5acb-a334-1d8de58489fc with identifier: 1788815
2025-12-05 20:01:39 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a8ade30f-b872-5acb-a334-1d8de58489fc with scrape type 1
2025-12-05 20:01:39 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item a8ade30f-b872-5acb-a334-1d8de58489fc
2025-12-05 20:01:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F499%252Faa7d661ac69f4fb5b1bd0abfd77a3501%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:39 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/499/aa7d661ac69f4fb5b1bd0abfd77a3501/images/large/1.jpg not downloaded yet
2025-12-05 20:01:39 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 2eda0255-6c15-52df-a582-06a3e26cd85b with identifier: 1788805
2025-12-05 20:01:39 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 2eda0255-6c15-52df-a582-06a3e26cd85b with identifier: 1788805
2025-12-05 20:01:39 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'renault' containing 'renault' and model 'master' containing 'master'
2025-12-05 20:01:39 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 2eda0255-6c15-52df-a582-06a3e26cd85b spider: ayvens.es
2025-12-05 20:01:39 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 2eda0255-6c15-52df-a582-06a3e26cd85b due to rule: makes_models_not_allowed.json, error: Rejected due to make 'renault' containing 'renault' and model 'master' containing 'master'
{'auction_title': '',
 'c_o_2_emission_value': 204,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '2.3L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '2eda0255-6c15-52df-a582-06a3e26cd85b',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'MASTER',
 'odometer': 30265,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_2eda0255-6c15-52df-a582-06a3e26cd85b/2eda0255-6c15-52df-a582-06a3e26cd85b_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 341692,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/586/fc53da839c5b4cd89d25318843373e53/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-04-29',
 'seats_number': 3,
 'seller_name': None,
 'title': 'RENAULT MASTER Furgon T L2H2 3500 dCi 96kW (130CV)',
 'transmission': 'manual',
 'trim': 'Furgon T L2H2 3500 dCi 96kW (130CV)',
 'vin': None,
 'year': 2019}
2025-12-05 20:01:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788805%27 HTTP/1.1" 200 None
2025-12-05 20:01:39 [ayvens.es] INFO: Saving data for 1788805: {'created_time': 1764964899.485738, 'last_price_update_time': 1764964899.485751, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788805') HTTP/1.1" 204 0
2025-12-05 20:01:39 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 561288b8-a2c7-5903-9453-25d19d8728ac with identifier: 1788806
2025-12-05 20:01:39 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 561288b8-a2c7-5903-9453-25d19d8728ac with identifier: 1788806
2025-12-05 20:01:39 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'peugeot', 'model': '2008', 'trim': 'active puretech 100 s&s bvm6', 'title': 'peugeot 2008 active puretech 100 s&s bvm6', 'price': 0, 'odometer': 58700, 'odometer_reading_unit': 'km', 'registration_date': '2020-11-16', 'year': 2020, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.2l', 'c_o_2_emission_value': 122, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '561288b8-a2c7-5903-9453-25d19d8728ac', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_561288b8-a2c7-5903-9453-25d19d8728ac/561288b8-a2c7-5903-9453-25d19d8728ac_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/234/d6a5c04fd82f4b0fbd670e2f8d3e134b/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 482520}], 'current_year_diff': 5, 'age_in_months': 60, 'make_model': 'peugeot 2008', 'sanitized_make': 'peugeot', 'odometer_in_km': 58700}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 60,
     'auction_title': '',
     'c_o_2_emission_value': 122,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.2l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': '561288b8-a2c7-5903-9453-25d19d8728ac',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'peugeot',
     'make_model': 'peugeot 2008',
     'model': '2008',
     'odometer': 58700,
     'odometer_in_km': 58700,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_561288b8-a2c7-5903-9453-25d19d8728ac/561288b8-a2c7-5903-9453-25d19d8728ac_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 482520,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/234/d6a5c04fd82f4b0fbd670e2f8d3e134b/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-11-16',
     'sanitized_make': 'peugeot',
     'seats_number': 5,
     'seller_name': None,
     'title': 'peugeot 2008 active puretech 100 s&s bvm6',
     'transmission': 'manual',
     'trim': 'active puretech 100 s&s bvm6',
     'vin': None,
     'year': 2020}
2025-12-05 20:01:39 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: PURETECH_FOR_COUNTRY failed for item: 561288b8-a2c7-5903-9453-25d19d8728ac spider: ayvens.es
2025-12-05 20:01:39 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 561288b8-a2c7-5903-9453-25d19d8728ac due to rule: puretech_for_country.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'peugeot', 'model': '2008', 'trim': 'active puretech 100 s&s bvm6', 'title': 'peugeot 2008 active puretech 100 s&s bvm6', 'price': 0, 'odometer': 58700, 'odometer_reading_unit': 'km', 'registration_date': '2020-11-16', 'year': 2020, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.2l', 'c_o_2_emission_value': 122, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '561288b8-a2c7-5903-9453-25d19d8728ac', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_561288b8-a2c7-5903-9453-25d19d8728ac/561288b8-a2c7-5903-9453-25d19d8728ac_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/234/d6a5c04fd82f4b0fbd670e2f8d3e134b/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 482520}], 'current_year_diff': 5, 'age_in_months': 60, 'make_model': 'peugeot 2008', 'sanitized_make': 'peugeot', 'odometer_in_km': 58700}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 60,
     'auction_title': '',
     'c_o_2_emission_value': 122,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.2l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': '561288b8-a2c7-5903-9453-25d19d8728ac',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'peugeot',
     'make_model': 'peugeot 2008',
     'model': '2008',
     'odometer': 58700,
     'odometer_in_km': 58700,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_561288b8-a2c7-5903-9453-25d19d8728ac/561288b8-a2c7-5903-9453-25d19d8728ac_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 482520,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/234/d6a5c04fd82f4b0fbd670e2f8d3e134b/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-11-16',
     'sanitized_make': 'peugeot',
     'seats_number': 5,
     'seller_name': None,
     'title': 'peugeot 2008 active puretech 100 s&s bvm6',
     'transmission': 'manual',
     'trim': 'active puretech 100 s&s bvm6',
     'vin': None,
     'year': 2020}
{'auction_title': '',
 'c_o_2_emission_value': 122,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.2L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '561288b8-a2c7-5903-9453-25d19d8728ac',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': '2008',
 'odometer': 58700,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_561288b8-a2c7-5903-9453-25d19d8728ac/561288b8-a2c7-5903-9453-25d19d8728ac_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 482520,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/234/d6a5c04fd82f4b0fbd670e2f8d3e134b/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-11-16',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT 2008 Active Puretech 100 S&S BVM6',
 'transmission': 'manual',
 'trim': 'Active Puretech 100 S&S BVM6',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788806%27 HTTP/1.1" 200 None
2025-12-05 20:01:39 [ayvens.es] INFO: Saving data for 1788806: {'created_time': 1764964899.870954, 'last_price_update_time': 1764964899.870963, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788806') HTTP/1.1" 204 0
2025-12-05 20:01:39 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/621/1bb5fa03ea864089b727092b766cf197/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:39 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/382/fd90fbecf1634afeb1c8ed4da32fc7de/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:39 [ayvens.es] INFO: Found listing with ID: 1788816
2025-12-05 20:01:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788816%27 HTTP/1.1" 200 None
2025-12-05 20:01:39 [ayvens.es] INFO: Scrape type for 1788816: 1
2025-12-05 20:01:39 [ayvens.es] INFO: Extracting data for car ID: 1788816
2025-12-05 20:01:39 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 22b6c1fc-2aa6-52d8-b95a-ee148dc3fd01 with identifier: 1788816
2025-12-05 20:01:39 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 22b6c1fc-2aa6-52d8-b95a-ee148dc3fd01 with scrape type 1
2025-12-05 20:01:39 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 22b6c1fc-2aa6-52d8-b95a-ee148dc3fd01
2025-12-05 20:01:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F494%252Ffeade0d02cd0445d9eafeecb9af53485%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:39 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/494/feade0d02cd0445d9eafeecb9af53485/images/large/1.jpg not downloaded yet
2025-12-05 20:01:39 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:39 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:39 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:39 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b06ad450>
2025-12-05 20:01:39 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:39 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:40 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:01:40 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:40 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:40 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:40 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_38b36fbb-cc49-5c85-9667-a272c908c0ef/38b36fbb-cc49-5c85-9667-a272c908c0ef_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F366%252F6c8ca8a3fdf94f3eaa113a62ed57c77a%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:40 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/366/6c8ca8a3fdf94f3eaa113a62ed57c77a/images/large/1.jpg saved to cache
2025-12-05 20:01:40 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 38b36fbb-cc49-5c85-9667-a272c908c0ef
2025-12-05 20:01:40 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 38b36fbb-cc49-5c85-9667-a272c908c0ef, skipping ID generation
2025-12-05 20:01:40 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 38b36fbb-cc49-5c85-9667-a272c908c0ef with scrape type 1
2025-12-05 20:01:40 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 38b36fbb-cc49-5c85-9667-a272c908c0ef from tracking dictionaries
2025-12-05 20:01:40 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 38b36fbb-cc49-5c85-9667-a272c908c0ef sending to next pipeline
2025-12-05 20:01:40 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 38b36fbb-cc49-5c85-9667-a272c908c0ef, identifier: 1788807
2025-12-05 20:01:40 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:40 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:40 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:40 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0d9be50>
2025-12-05 20:01:40 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:40 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:40 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:01:40 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:40 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:40 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:40 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_35150818-ecf9-5843-840c-0b67f3c2ac3c/35150818-ecf9-5843-840c-0b67f3c2ac3c_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F382%252Fc51d6af8a84646b1a411004c463671f3%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:40 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/382/c51d6af8a84646b1a411004c463671f3/images/large/1.jpg saved to cache
2025-12-05 20:01:40 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 35150818-ecf9-5843-840c-0b67f3c2ac3c
2025-12-05 20:01:40 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 35150818-ecf9-5843-840c-0b67f3c2ac3c, skipping ID generation
2025-12-05 20:01:40 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 35150818-ecf9-5843-840c-0b67f3c2ac3c with scrape type 1
2025-12-05 20:01:40 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 35150818-ecf9-5843-840c-0b67f3c2ac3c from tracking dictionaries
2025-12-05 20:01:40 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 35150818-ecf9-5843-840c-0b67f3c2ac3c sending to next pipeline
2025-12-05 20:01:40 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 35150818-ecf9-5843-840c-0b67f3c2ac3c, identifier: 1788808
2025-12-05 20:01:40 [ayvens.es] INFO: Found listing with ID: 1788817
2025-12-05 20:01:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788817%27 HTTP/1.1" 200 None
2025-12-05 20:01:40 [ayvens.es] INFO: Scrape type for 1788817: 1
2025-12-05 20:01:40 [ayvens.es] INFO: Extracting data for car ID: 1788817
2025-12-05 20:01:40 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: a65f649a-65d3-5530-add8-8dd277c89621 with identifier: 1788817
2025-12-05 20:01:40 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a65f649a-65d3-5530-add8-8dd277c89621 with scrape type 1
2025-12-05 20:01:40 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item a65f649a-65d3-5530-add8-8dd277c89621
2025-12-05 20:01:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F044%252F776362cac17349ecaedfdeb17b8964fd%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:40 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/044/776362cac17349ecaedfdeb17b8964fd/images/large/1.jpg not downloaded yet
2025-12-05 20:01:40 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 38b36fbb-cc49-5c85-9667-a272c908c0ef with identifier: 1788807
2025-12-05 20:01:40 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 38b36fbb-cc49-5c85-9667-a272c908c0ef with identifier: 1788807
2025-12-05 20:01:41 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 38b36fbb-cc49-5c85-9667-a272c908c0ef with identifier: 1788807 to the API
2025-12-05 20:01:41 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:41 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:41 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/366/6c8ca8a3fdf94f3eaa113a62ed57c77a/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 152,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.6L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '38b36fbb-cc49-5c85-9667-a272c908c0ef',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'FIAT',
 'model': '500X',
 'odometer': 92835,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_38b36fbb-cc49-5c85-9667-a272c908c0ef/38b36fbb-cc49-5c85-9667-a272c908c0ef_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 320803,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/366/6c8ca8a3fdf94f3eaa113a62ed57c77a/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-04-22',
 'seats_number': 5,
 'seller_name': None,
 'title': 'FIAT 500X Urban 1.6 E-Torq 81KW (110 CV) 4x2 S&S',
 'transmission': 'manual',
 'trim': 'Urban 1.6 E-Torq 81KW (110 CV) 4x2 S&S',
 'vin': None,
 'year': 2019}
2025-12-05 20:01:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788807%27 HTTP/1.1" 200 None
2025-12-05 20:01:41 [ayvens.es] INFO: Saving data for 1788807: {'created_time': 1764964901.298385, 'last_price_update_time': 1764964901.298397, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788807') HTTP/1.1" 204 0
2025-12-05 20:01:41 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 35150818-ecf9-5843-840c-0b67f3c2ac3c with identifier: 1788808
2025-12-05 20:01:41 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 35150818-ecf9-5843-840c-0b67f3c2ac3c with identifier: 1788808
2025-12-05 20:01:41 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 35150818-ecf9-5843-840c-0b67f3c2ac3c with identifier: 1788808 to the API
2025-12-05 20:01:41 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:41 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:41 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/382/c51d6af8a84646b1a411004c463671f3/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 126,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '35150818-ecf9-5843-840c-0b67f3c2ac3c',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'CUPRA',
 'model': 'FORMENTOR',
 'odometer': 46955,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_35150818-ecf9-5843-840c-0b67f3c2ac3c/35150818-ecf9-5843-840c-0b67f3c2ac3c_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 377052,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/382/c51d6af8a84646b1a411004c463671f3/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-12-20',
 'seats_number': 5,
 'seller_name': None,
 'title': 'CUPRA FORMENTOR 2.0 TDI 110kW (150 CV)',
 'transmission': 'manual',
 'trim': '2.0 TDI 110kW (150 CV)',
 'vin': None,
 'year': 2021}
2025-12-05 20:01:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788808%27 HTTP/1.1" 200 None
2025-12-05 20:01:41 [ayvens.es] INFO: Saving data for 1788808: {'created_time': 1764964901.695756, 'last_price_update_time': 1764964901.695765, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788808') HTTP/1.1" 204 0
2025-12-05 20:01:41 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/640/4d70e3c1bd1f447098a863661f2fd6bc/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:41 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/455/d47c0ac527294bdeae6e23031d0625de/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:41 [ayvens.es] INFO: Found listing with ID: 1788818
2025-12-05 20:01:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788818%27 HTTP/1.1" 200 None
2025-12-05 20:01:41 [ayvens.es] INFO: Scrape type for 1788818: 1
2025-12-05 20:01:41 [ayvens.es] INFO: Extracting data for car ID: 1788818
2025-12-05 20:01:41 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 8d504324-e6a9-51f1-864e-25267568715d with identifier: 1788818
2025-12-05 20:01:41 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 8d504324-e6a9-51f1-864e-25267568715d with scrape type 1
2025-12-05 20:01:41 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 8d504324-e6a9-51f1-864e-25267568715d
2025-12-05 20:01:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F557%252Ffeddd4bc7acd402ab992dcf17c5634a2%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:41 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/557/feddd4bc7acd402ab992dcf17c5634a2/images/large/1.jpg not downloaded yet
2025-12-05 20:01:41 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:41 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:41 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:41 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0d5ded0>
2025-12-05 20:01:41 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:41 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:41 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:01:41 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:42 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:42 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:42 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_5798ae1e-5084-5fe0-929e-173309c1b413/5798ae1e-5084-5fe0-929e-173309c1b413_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F621%252F1bb5fa03ea864089b727092b766cf197%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:42 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/621/1bb5fa03ea864089b727092b766cf197/images/large/1.jpg saved to cache
2025-12-05 20:01:42 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 5798ae1e-5084-5fe0-929e-173309c1b413
2025-12-05 20:01:42 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 5798ae1e-5084-5fe0-929e-173309c1b413, skipping ID generation
2025-12-05 20:01:42 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 5798ae1e-5084-5fe0-929e-173309c1b413 with scrape type 1
2025-12-05 20:01:42 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 5798ae1e-5084-5fe0-929e-173309c1b413 from tracking dictionaries
2025-12-05 20:01:42 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 5798ae1e-5084-5fe0-929e-173309c1b413 sending to next pipeline
2025-12-05 20:01:42 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 5798ae1e-5084-5fe0-929e-173309c1b413, identifier: 1788809
2025-12-05 20:01:42 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:42 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:42 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:42 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0450310>
2025-12-05 20:01:42 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:42 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:42 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:01:42 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:42 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:42 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:42 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_6af472fb-bac7-5b7b-b8fb-5e2917a08a62/6af472fb-bac7-5b7b-b8fb-5e2917a08a62_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F382%252Ffd90fbecf1634afeb1c8ed4da32fc7de%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:42 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/382/fd90fbecf1634afeb1c8ed4da32fc7de/images/large/1.jpg saved to cache
2025-12-05 20:01:42 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 6af472fb-bac7-5b7b-b8fb-5e2917a08a62
2025-12-05 20:01:42 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 6af472fb-bac7-5b7b-b8fb-5e2917a08a62, skipping ID generation
2025-12-05 20:01:42 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6af472fb-bac7-5b7b-b8fb-5e2917a08a62 with scrape type 1
2025-12-05 20:01:42 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 6af472fb-bac7-5b7b-b8fb-5e2917a08a62 from tracking dictionaries
2025-12-05 20:01:42 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 6af472fb-bac7-5b7b-b8fb-5e2917a08a62 sending to next pipeline
2025-12-05 20:01:42 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 6af472fb-bac7-5b7b-b8fb-5e2917a08a62, identifier: 1788810
2025-12-05 20:01:42 [ayvens.es] INFO: Found listing with ID: 1788819
2025-12-05 20:01:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788819%27 HTTP/1.1" 200 None
2025-12-05 20:01:42 [ayvens.es] INFO: Scrape type for 1788819: 1
2025-12-05 20:01:42 [ayvens.es] INFO: Extracting data for car ID: 1788819
2025-12-05 20:01:42 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 0031150e-6f50-5280-840a-608ad907f154 with identifier: 1788819
2025-12-05 20:01:42 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 0031150e-6f50-5280-840a-608ad907f154 with scrape type 1
2025-12-05 20:01:42 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 0031150e-6f50-5280-840a-608ad907f154
2025-12-05 20:01:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F535%252F0688f503ef384c568fd0ceff9f4364ce%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:42 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/535/0688f503ef384c568fd0ceff9f4364ce/images/large/1.jpg not downloaded yet
2025-12-05 20:01:42 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 5798ae1e-5084-5fe0-929e-173309c1b413 with identifier: 1788809
2025-12-05 20:01:42 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 5798ae1e-5084-5fe0-929e-173309c1b413 with identifier: 1788809
2025-12-05 20:01:42 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 5798ae1e-5084-5fe0-929e-173309c1b413 with identifier: 1788809 to the API
2025-12-05 20:01:42 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:42 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:43 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/621/1bb5fa03ea864089b727092b766cf197/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 152,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.1L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '5798ae1e-5084-5fe0-929e-173309c1b413',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'ALFA ROMEO',
 'model': 'STELVIO',
 'odometer': 28870,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_5798ae1e-5084-5fe0-929e-173309c1b413/5798ae1e-5084-5fe0-929e-173309c1b413_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 298507,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/621/1bb5fa03ea864089b727092b766cf197/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2023-12-20',
 'seats_number': 5,
 'seller_name': None,
 'title': 'ALFA ROMEO STELVIO 2.2 Diesel 118kW (160CV) SPRINT RWD',
 'transmission': 'manual',
 'trim': '2.2 Diesel 118kW (160CV) SPRINT RWD',
 'vin': None,
 'year': 2023}
2025-12-05 20:01:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788809%27 HTTP/1.1" 200 None
2025-12-05 20:01:43 [ayvens.es] INFO: Saving data for 1788809: {'created_time': 1764964903.042204, 'last_price_update_time': 1764964903.042216, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788809') HTTP/1.1" 204 0
2025-12-05 20:01:43 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 6af472fb-bac7-5b7b-b8fb-5e2917a08a62 with identifier: 1788810
2025-12-05 20:01:43 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 6af472fb-bac7-5b7b-b8fb-5e2917a08a62 with identifier: 1788810
2025-12-05 20:01:43 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: 167831 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    167831
2025-12-05 20:01:43 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE_FOR_COUNTRY failed for item: 6af472fb-bac7-5b7b-b8fb-5e2917a08a62 spider: ayvens.es
2025-12-05 20:01:43 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 6af472fb-bac7-5b7b-b8fb-5e2917a08a62 due to rule: high_mileage_for_country.json, error: 167831 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    167831
{'auction_title': '',
 'c_o_2_emission_value': 135,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.9L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '6af472fb-bac7-5b7b-b8fb-5e2917a08a62',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'MERCEDES-BENZ',
 'model': 'B-CLASS',
 'odometer': 167831,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_6af472fb-bac7-5b7b-b8fb-5e2917a08a62/6af472fb-bac7-5b7b-b8fb-5e2917a08a62_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 347878,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/382/fd90fbecf1634afeb1c8ed4da32fc7de/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-05-26',
 'seats_number': 5,
 'seller_name': None,
 'title': 'MERCEDES-BENZ B-CLASS B 180 d',
 'transmission': 'manual',
 'trim': 'B 180 d',
 'vin': None,
 'year': 2021}
2025-12-05 20:01:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788810%27 HTTP/1.1" 200 None
2025-12-05 20:01:43 [ayvens.es] INFO: Saving data for 1788810: {'created_time': 1764964903.292508, 'last_price_update_time': 1764964903.292518, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788810') HTTP/1.1" 204 0
2025-12-05 20:01:43 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/530/4bd8594581704e06a0412301438ec582/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:43 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/903/b4e1f4cd501b472e91bcfcb1c6ea17ac/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:43 [ayvens.es] INFO: Found listing with ID: 1788820
2025-12-05 20:01:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788820%27 HTTP/1.1" 200 None
2025-12-05 20:01:43 [ayvens.es] INFO: Scrape type for 1788820: 1
2025-12-05 20:01:43 [ayvens.es] INFO: Extracting data for car ID: 1788820
2025-12-05 20:01:43 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: e9c935b9-07e1-5fd7-8345-0ef36b38837f with identifier: 1788820
2025-12-05 20:01:43 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e9c935b9-07e1-5fd7-8345-0ef36b38837f with scrape type 1
2025-12-05 20:01:43 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item e9c935b9-07e1-5fd7-8345-0ef36b38837f
2025-12-05 20:01:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F682%252Fa2b71027dab949c884665b498e472086%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:43 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/682/a2b71027dab949c884665b498e472086/images/large/1.jpg not downloaded yet
2025-12-05 20:01:43 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:43 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:43 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:43 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0af4150>
2025-12-05 20:01:43 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:43 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:43 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:43 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:43 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:43 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:43 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_6492ac7c-2210-5950-b04d-43b96b7f4cbf/6492ac7c-2210-5950-b04d-43b96b7f4cbf_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F640%252F4d70e3c1bd1f447098a863661f2fd6bc%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:43 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/640/4d70e3c1bd1f447098a863661f2fd6bc/images/large/1.jpg saved to cache
2025-12-05 20:01:43 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 6492ac7c-2210-5950-b04d-43b96b7f4cbf
2025-12-05 20:01:43 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 6492ac7c-2210-5950-b04d-43b96b7f4cbf, skipping ID generation
2025-12-05 20:01:43 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6492ac7c-2210-5950-b04d-43b96b7f4cbf with scrape type 1
2025-12-05 20:01:43 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 6492ac7c-2210-5950-b04d-43b96b7f4cbf from tracking dictionaries
2025-12-05 20:01:43 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 6492ac7c-2210-5950-b04d-43b96b7f4cbf sending to next pipeline
2025-12-05 20:01:43 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 6492ac7c-2210-5950-b04d-43b96b7f4cbf, identifier: 1788812
2025-12-05 20:01:43 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:43 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:43 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:43 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b0c0d0>
2025-12-05 20:01:43 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:43 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:43 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:43 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:44 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:44 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:44 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_7ba0cbb9-0036-5f93-af73-83b770053d0c/7ba0cbb9-0036-5f93-af73-83b770053d0c_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F455%252Fd47c0ac527294bdeae6e23031d0625de%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:44 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/455/d47c0ac527294bdeae6e23031d0625de/images/large/1.jpg saved to cache
2025-12-05 20:01:44 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 7ba0cbb9-0036-5f93-af73-83b770053d0c
2025-12-05 20:01:44 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 7ba0cbb9-0036-5f93-af73-83b770053d0c, skipping ID generation
2025-12-05 20:01:44 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 7ba0cbb9-0036-5f93-af73-83b770053d0c with scrape type 1
2025-12-05 20:01:44 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 7ba0cbb9-0036-5f93-af73-83b770053d0c from tracking dictionaries
2025-12-05 20:01:44 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 7ba0cbb9-0036-5f93-af73-83b770053d0c sending to next pipeline
2025-12-05 20:01:44 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 7ba0cbb9-0036-5f93-af73-83b770053d0c, identifier: 1788811
2025-12-05 20:01:44 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 6492ac7c-2210-5950-b04d-43b96b7f4cbf with identifier: 1788812
2025-12-05 20:01:44 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 6492ac7c-2210-5950-b04d-43b96b7f4cbf with identifier: 1788812
2025-12-05 20:01:44 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 6492ac7c-2210-5950-b04d-43b96b7f4cbf with identifier: 1788812 to the API
2025-12-05 20:01:44 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:44 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/640/4d70e3c1bd1f447098a863661f2fd6bc/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 98,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '6492ac7c-2210-5950-b04d-43b96b7f4cbf',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'AUDI',
 'model': 'A1',
 'odometer': 121183,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_6492ac7c-2210-5950-b04d-43b96b7f4cbf/6492ac7c-2210-5950-b04d-43b96b7f4cbf_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 325786,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/640/4d70e3c1bd1f447098a863661f2fd6bc/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2018-06-22',
 'seats_number': 4,
 'seller_name': None,
 'title': 'AUDI A1 Adrenalin 1.0 TFSI 70kW (95CV) Sportback',
 'transmission': 'manual',
 'trim': 'Adrenalin 1.0 TFSI 70kW (95CV) Sportback',
 'vin': None,
 'year': 2018}
2025-12-05 20:01:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788812%27 HTTP/1.1" 200 None
2025-12-05 20:01:44 [ayvens.es] INFO: Saving data for 1788812: {'created_time': 1764964904.759977, 'last_price_update_time': 1764964904.759986, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788812') HTTP/1.1" 204 0
2025-12-05 20:01:44 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 7ba0cbb9-0036-5f93-af73-83b770053d0c with identifier: 1788811
2025-12-05 20:01:44 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 7ba0cbb9-0036-5f93-af73-83b770053d0c with identifier: 1788811
2025-12-05 20:01:44 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 7ba0cbb9-0036-5f93-af73-83b770053d0c with identifier: 1788811 to the API
2025-12-05 20:01:44 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:45 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/455/d47c0ac527294bdeae6e23031d0625de/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 147,
 'color': 'Serie',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '7ba0cbb9-0036-5f93-af73-83b770053d0c',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SKODA',
 'model': 'KAROQ',
 'odometer': 149261,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_7ba0cbb9-0036-5f93-af73-83b770053d0c/7ba0cbb9-0036-5f93-af73-83b770053d0c_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 350489,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/455/d47c0ac527294bdeae6e23031d0625de/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2022-01-31',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SKODA KAROQ 2.0 TDI 110kW (150CV) DSG 4X4 Ambition',
 'transmission': 'manual',
 'trim': '2.0 TDI 110kW (150CV) DSG 4X4 Ambition',
 'vin': None,
 'year': 2022}
2025-12-05 20:01:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788811%27 HTTP/1.1" 200 None
2025-12-05 20:01:45 [ayvens.es] INFO: Saving data for 1788811: {'created_time': 1764964905.146123, 'last_price_update_time': 1764964905.146132, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788811') HTTP/1.1" 204 0
2025-12-05 20:01:45 [ayvens.es] INFO: Found listing with ID: 1788821
2025-12-05 20:01:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788821%27 HTTP/1.1" 200 None
2025-12-05 20:01:45 [ayvens.es] INFO: Scrape type for 1788821: 1
2025-12-05 20:01:45 [ayvens.es] INFO: Extracting data for car ID: 1788821
2025-12-05 20:01:45 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 3ae49fb1-7c59-5361-a23e-d7f95bb9ba88 with identifier: 1788821
2025-12-05 20:01:45 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 3ae49fb1-7c59-5361-a23e-d7f95bb9ba88 with scrape type 1
2025-12-05 20:01:45 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 3ae49fb1-7c59-5361-a23e-d7f95bb9ba88
2025-12-05 20:01:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F923%252Fe6db74e18b0f4c80b047e2d39b3cb602%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:45 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/923/e6db74e18b0f4c80b047e2d39b3cb602/images/large/1.jpg not downloaded yet
2025-12-05 20:01:45 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:45 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:45 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:45 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b9a150>
2025-12-05 20:01:45 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:45 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:45 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:01:45 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:45 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:45 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:45 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_1d5606eb-fcc8-5c67-ad8c-4f32d53905eb/1d5606eb-fcc8-5c67-ad8c-4f32d53905eb_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F530%252F4bd8594581704e06a0412301438ec582%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:45 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/530/4bd8594581704e06a0412301438ec582/images/large/1.jpg saved to cache
2025-12-05 20:01:45 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 1d5606eb-fcc8-5c67-ad8c-4f32d53905eb
2025-12-05 20:01:45 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 1d5606eb-fcc8-5c67-ad8c-4f32d53905eb, skipping ID generation
2025-12-05 20:01:45 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 1d5606eb-fcc8-5c67-ad8c-4f32d53905eb with scrape type 1
2025-12-05 20:01:45 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 1d5606eb-fcc8-5c67-ad8c-4f32d53905eb from tracking dictionaries
2025-12-05 20:01:45 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 1d5606eb-fcc8-5c67-ad8c-4f32d53905eb sending to next pipeline
2025-12-05 20:01:45 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 1d5606eb-fcc8-5c67-ad8c-4f32d53905eb, identifier: 1788814
2025-12-05 20:01:45 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:45 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:45 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:45 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b98550>
2025-12-05 20:01:45 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:45 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:45 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:01:45 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:45 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:45 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:46 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_2fe30fc3-4c86-5e5a-9db0-652dec8cc6b6/2fe30fc3-4c86-5e5a-9db0-652dec8cc6b6_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F903%252Fb4e1f4cd501b472e91bcfcb1c6ea17ac%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:46 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/903/b4e1f4cd501b472e91bcfcb1c6ea17ac/images/large/1.jpg saved to cache
2025-12-05 20:01:46 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 2fe30fc3-4c86-5e5a-9db0-652dec8cc6b6
2025-12-05 20:01:46 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 2fe30fc3-4c86-5e5a-9db0-652dec8cc6b6, skipping ID generation
2025-12-05 20:01:46 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 2fe30fc3-4c86-5e5a-9db0-652dec8cc6b6 with scrape type 1
2025-12-05 20:01:46 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 2fe30fc3-4c86-5e5a-9db0-652dec8cc6b6 from tracking dictionaries
2025-12-05 20:01:46 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 2fe30fc3-4c86-5e5a-9db0-652dec8cc6b6 sending to next pipeline
2025-12-05 20:01:46 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 2fe30fc3-4c86-5e5a-9db0-652dec8cc6b6, identifier: 1788813
2025-12-05 20:01:46 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/494/feade0d02cd0445d9eafeecb9af53485/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:46 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/499/aa7d661ac69f4fb5b1bd0abfd77a3501/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:46 [ayvens.es] INFO: Found listing with ID: 1788822
2025-12-05 20:01:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788822%27 HTTP/1.1" 200 None
2025-12-05 20:01:46 [ayvens.es] INFO: Scrape type for 1788822: 1
2025-12-05 20:01:46 [ayvens.es] INFO: Extracting data for car ID: 1788822
2025-12-05 20:01:46 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 684d079c-ea8e-51e8-bcd7-87404bbefcc4 with identifier: 1788822
2025-12-05 20:01:46 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 684d079c-ea8e-51e8-bcd7-87404bbefcc4 with scrape type 1
2025-12-05 20:01:46 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 684d079c-ea8e-51e8-bcd7-87404bbefcc4
2025-12-05 20:01:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F344%252F14af9ebacd8046d79a147eae1e0b25f4%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:46 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/344/14af9ebacd8046d79a147eae1e0b25f4/images/large/1.jpg not downloaded yet
2025-12-05 20:01:46 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 1d5606eb-fcc8-5c67-ad8c-4f32d53905eb with identifier: 1788814
2025-12-05 20:01:46 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 1d5606eb-fcc8-5c67-ad8c-4f32d53905eb with identifier: 1788814
2025-12-05 20:01:46 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 1d5606eb-fcc8-5c67-ad8c-4f32d53905eb with identifier: 1788814 to the API
2025-12-05 20:01:46 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:46 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:46 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/530/4bd8594581704e06a0412301438ec582/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 109,
 'color': 'Negro',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.6L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '1d5606eb-fcc8-5c67-ad8c-4f32d53905eb',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'PASSAT',
 'odometer': 113936,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_1d5606eb-fcc8-5c67-ad8c-4f32d53905eb/1d5606eb-fcc8-5c67-ad8c-4f32d53905eb_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 345171,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/530/4bd8594581704e06a0412301438ec582/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-06-30',
 'seats_number': 5,
 'seller_name': None,
 'title': 'VOLKSWAGEN PASSAT Executive 1.6 TDI 88kW (120CV) DSG',
 'transmission': 'manual',
 'trim': 'Executive 1.6 TDI 88kW (120CV) DSG',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788814%27 HTTP/1.1" 200 None
2025-12-05 20:01:46 [ayvens.es] INFO: Saving data for 1788814: {'created_time': 1764964906.593055, 'last_price_update_time': 1764964906.593065, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788814') HTTP/1.1" 204 0
2025-12-05 20:01:46 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 2fe30fc3-4c86-5e5a-9db0-652dec8cc6b6 with identifier: 1788813
2025-12-05 20:01:46 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 2fe30fc3-4c86-5e5a-9db0-652dec8cc6b6 with identifier: 1788813
2025-12-05 20:01:46 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 2fe30fc3-4c86-5e5a-9db0-652dec8cc6b6 with identifier: 1788813 to the API
2025-12-05 20:01:46 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:46 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:46 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/903/b4e1f4cd501b472e91bcfcb1c6ea17ac/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 76,
 'color': 'Rojo',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.8L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'hybrid',
 'id': '2fe30fc3-4c86-5e5a-9db0-652dec8cc6b6',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'TOYOTA',
 'model': 'COROLLA',
 'odometer': 116600,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_2fe30fc3-4c86-5e5a-9db0-652dec8cc6b6/2fe30fc3-4c86-5e5a-9db0-652dec8cc6b6_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 383809,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/903/b4e1f4cd501b472e91bcfcb1c6ea17ac/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-01-24',
 'seats_number': 5,
 'seller_name': None,
 'title': 'TOYOTA COROLLA 1.8 125H ACTIVE TECH E-CVT',
 'transmission': 'manual',
 'trim': '1.8 125H ACTIVE TECH E-CVT',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788813%27 HTTP/1.1" 200 None
2025-12-05 20:01:46 [ayvens.es] INFO: Saving data for 1788813: {'created_time': 1764964906.964572, 'last_price_update_time': 1764964906.964582, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788813') HTTP/1.1" 204 0
2025-12-05 20:01:46 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/557/feddd4bc7acd402ab992dcf17c5634a2/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:46 [ayvens.es] INFO: Found listing with ID: 1788823
2025-12-05 20:01:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788823%27 HTTP/1.1" 200 None
2025-12-05 20:01:47 [ayvens.es] INFO: Scrape type for 1788823: 1
2025-12-05 20:01:47 [ayvens.es] INFO: Extracting data for car ID: 1788823
2025-12-05 20:01:47 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 8f37213a-fe32-561d-be2b-ba106a9c304f with identifier: 1788823
2025-12-05 20:01:47 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 8f37213a-fe32-561d-be2b-ba106a9c304f with scrape type 1
2025-12-05 20:01:47 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 8f37213a-fe32-561d-be2b-ba106a9c304f
2025-12-05 20:01:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F873%252F80e49a04c57f47e98181699f4f1e7bf1%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:47 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/873/80e49a04c57f47e98181699f4f1e7bf1/images/large/1.jpg not downloaded yet
2025-12-05 20:01:47 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/044/776362cac17349ecaedfdeb17b8964fd/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:47 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:47 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:47 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:47 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b28ec290>
2025-12-05 20:01:47 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:47 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:47 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:01:47 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:47 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:47 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:47 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_22b6c1fc-2aa6-52d8-b95a-ee148dc3fd01/22b6c1fc-2aa6-52d8-b95a-ee148dc3fd01_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F494%252Ffeade0d02cd0445d9eafeecb9af53485%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:47 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/494/feade0d02cd0445d9eafeecb9af53485/images/large/1.jpg saved to cache
2025-12-05 20:01:47 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 22b6c1fc-2aa6-52d8-b95a-ee148dc3fd01
2025-12-05 20:01:47 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 22b6c1fc-2aa6-52d8-b95a-ee148dc3fd01, skipping ID generation
2025-12-05 20:01:47 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 22b6c1fc-2aa6-52d8-b95a-ee148dc3fd01 with scrape type 1
2025-12-05 20:01:47 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 22b6c1fc-2aa6-52d8-b95a-ee148dc3fd01 from tracking dictionaries
2025-12-05 20:01:47 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 22b6c1fc-2aa6-52d8-b95a-ee148dc3fd01 sending to next pipeline
2025-12-05 20:01:47 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 22b6c1fc-2aa6-52d8-b95a-ee148dc3fd01, identifier: 1788816
2025-12-05 20:01:47 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:47 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:47 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:47 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b04b35d0>
2025-12-05 20:01:47 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:47 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:47 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:47 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:47 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:47 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:47 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_a8ade30f-b872-5acb-a334-1d8de58489fc/a8ade30f-b872-5acb-a334-1d8de58489fc_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F499%252Faa7d661ac69f4fb5b1bd0abfd77a3501%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:47 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/499/aa7d661ac69f4fb5b1bd0abfd77a3501/images/large/1.jpg saved to cache
2025-12-05 20:01:47 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item a8ade30f-b872-5acb-a334-1d8de58489fc
2025-12-05 20:01:47 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: a8ade30f-b872-5acb-a334-1d8de58489fc, skipping ID generation
2025-12-05 20:01:47 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a8ade30f-b872-5acb-a334-1d8de58489fc with scrape type 1
2025-12-05 20:01:47 [crawlers.middlewares.photo_download_middleware] INFO: Removing item a8ade30f-b872-5acb-a334-1d8de58489fc from tracking dictionaries
2025-12-05 20:01:47 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item a8ade30f-b872-5acb-a334-1d8de58489fc sending to next pipeline
2025-12-05 20:01:48 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: a8ade30f-b872-5acb-a334-1d8de58489fc, identifier: 1788815
2025-12-05 20:01:48 [ayvens.es] INFO: Found listing with ID: 1788824
2025-12-05 20:01:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788824%27 HTTP/1.1" 200 None
2025-12-05 20:01:48 [ayvens.es] INFO: Scrape type for 1788824: 1
2025-12-05 20:01:48 [ayvens.es] INFO: Extracting data for car ID: 1788824
2025-12-05 20:01:48 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 56720714-2e01-55b3-81e9-3955f7341147 with identifier: 1788824
2025-12-05 20:01:48 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 56720714-2e01-55b3-81e9-3955f7341147 with scrape type 1
2025-12-05 20:01:48 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 56720714-2e01-55b3-81e9-3955f7341147
2025-12-05 20:01:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F336%252F79500f829c4f4abfb9ef79f6488c3f77%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:48 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/336/79500f829c4f4abfb9ef79f6488c3f77/images/large/1.jpg not downloaded yet
2025-12-05 20:01:48 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 22b6c1fc-2aa6-52d8-b95a-ee148dc3fd01 with identifier: 1788816
2025-12-05 20:01:48 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:01:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:01:48 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 22b6c1fc-2aa6-52d8-b95a-ee148dc3fd01 with identifier: 1788816
2025-12-05 20:01:48 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 22b6c1fc-2aa6-52d8-b95a-ee148dc3fd01 with identifier: 1788816 to the API
2025-12-05 20:01:48 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:48 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:48 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/494/feade0d02cd0445d9eafeecb9af53485/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 145,
 'color': 'Dark',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.6L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'hybrid',
 'id': '22b6c1fc-2aa6-52d8-b95a-ee148dc3fd01',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'HYUNDAI',
 'model': 'TUCSON',
 'odometer': 59184,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_22b6c1fc-2aa6-52d8-b95a-ee148dc3fd01/22b6c1fc-2aa6-52d8-b95a-ee148dc3fd01_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 376596,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/494/feade0d02cd0445d9eafeecb9af53485/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-10-06',
 'seats_number': 5,
 'seller_name': None,
 'title': 'HYUNDAI TUCSON 1.6 TGDI 169kW (230CV) HEV Style Aut 4x4',
 'transmission': 'manual',
 'trim': '1.6 TGDI 169kW (230CV) HEV Style Aut 4x4',
 'vin': None,
 'year': 2021}
2025-12-05 20:01:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788816%27 HTTP/1.1" 200 None
2025-12-05 20:01:48 [ayvens.es] INFO: Saving data for 1788816: {'created_time': 1764964908.513186, 'last_price_update_time': 1764964908.513195, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788816') HTTP/1.1" 204 0
2025-12-05 20:01:48 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: a8ade30f-b872-5acb-a334-1d8de58489fc with identifier: 1788815
2025-12-05 20:01:48 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: a8ade30f-b872-5acb-a334-1d8de58489fc with identifier: 1788815
2025-12-05 20:01:48 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: a8ade30f-b872-5acb-a334-1d8de58489fc with identifier: 1788815 to the API
2025-12-05 20:01:48 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:48 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:48 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/499/aa7d661ac69f4fb5b1bd0abfd77a3501/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 106,
 'color': 'Serie',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'a8ade30f-b872-5acb-a334-1d8de58489fc',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': '208',
 'odometer': 87970,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_a8ade30f-b872-5acb-a334-1d8de58489fc/a8ade30f-b872-5acb-a334-1d8de58489fc_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 356848,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/499/aa7d661ac69f4fb5b1bd0abfd77a3501/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2022-01-31',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT 208 BlueHDi 73kW (100CV) Active',
 'transmission': 'manual',
 'trim': 'BlueHDi 73kW (100CV) Active',
 'vin': None,
 'year': 2022}
2025-12-05 20:01:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788815%27 HTTP/1.1" 200 None
2025-12-05 20:01:48 [ayvens.es] INFO: Saving data for 1788815: {'created_time': 1764964908.887268, 'last_price_update_time': 1764964908.887279, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788815') HTTP/1.1" 204 0
2025-12-05 20:01:48 [ayvens.es] INFO: Found listing with ID: 1788825
2025-12-05 20:01:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788825%27 HTTP/1.1" 200 None
2025-12-05 20:01:48 [ayvens.es] INFO: Scrape type for 1788825: 1
2025-12-05 20:01:48 [ayvens.es] INFO: Extracting data for car ID: 1788825
2025-12-05 20:01:48 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 25f0daf4-d28a-5beb-84ab-9bef8b8588ea with identifier: 1788825
2025-12-05 20:01:48 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 25f0daf4-d28a-5beb-84ab-9bef8b8588ea with scrape type 1
2025-12-05 20:01:48 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 25f0daf4-d28a-5beb-84ab-9bef8b8588ea
2025-12-05 20:01:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F303%252F4f120c8b31484352b5f3370ab017c5b2%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:48 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/303/4f120c8b31484352b5f3370ab017c5b2/images/large/1.jpg not downloaded yet
2025-12-05 20:01:48 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/682/a2b71027dab949c884665b498e472086/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:48 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:48 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:48 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:48 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b4cad0>
2025-12-05 20:01:48 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:49 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:49 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:01:49 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:49 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:49 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:49 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_8d504324-e6a9-51f1-864e-25267568715d/8d504324-e6a9-51f1-864e-25267568715d_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F557%252Ffeddd4bc7acd402ab992dcf17c5634a2%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:49 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/557/feddd4bc7acd402ab992dcf17c5634a2/images/large/1.jpg saved to cache
2025-12-05 20:01:49 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 8d504324-e6a9-51f1-864e-25267568715d
2025-12-05 20:01:49 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 8d504324-e6a9-51f1-864e-25267568715d, skipping ID generation
2025-12-05 20:01:49 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 8d504324-e6a9-51f1-864e-25267568715d with scrape type 1
2025-12-05 20:01:49 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 8d504324-e6a9-51f1-864e-25267568715d from tracking dictionaries
2025-12-05 20:01:49 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 8d504324-e6a9-51f1-864e-25267568715d sending to next pipeline
2025-12-05 20:01:49 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 8d504324-e6a9-51f1-864e-25267568715d, identifier: 1788818
2025-12-05 20:01:49 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:49 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:49 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:49 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b58550>
2025-12-05 20:01:49 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:49 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:49 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1969
2025-12-05 20:01:49 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:49 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:49 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:49 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_a65f649a-65d3-5530-add8-8dd277c89621/a65f649a-65d3-5530-add8-8dd277c89621_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F044%252F776362cac17349ecaedfdeb17b8964fd%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:49 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/044/776362cac17349ecaedfdeb17b8964fd/images/large/1.jpg saved to cache
2025-12-05 20:01:49 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item a65f649a-65d3-5530-add8-8dd277c89621
2025-12-05 20:01:49 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: a65f649a-65d3-5530-add8-8dd277c89621, skipping ID generation
2025-12-05 20:01:49 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a65f649a-65d3-5530-add8-8dd277c89621 with scrape type 1
2025-12-05 20:01:49 [crawlers.middlewares.photo_download_middleware] INFO: Removing item a65f649a-65d3-5530-add8-8dd277c89621 from tracking dictionaries
2025-12-05 20:01:49 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item a65f649a-65d3-5530-add8-8dd277c89621 sending to next pipeline
2025-12-05 20:01:49 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: a65f649a-65d3-5530-add8-8dd277c89621, identifier: 1788817
2025-12-05 20:01:49 [ayvens.es] INFO: Found listing with ID: 1788826
2025-12-05 20:01:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788826%27 HTTP/1.1" 200 None
2025-12-05 20:01:49 [ayvens.es] INFO: Scrape type for 1788826: 1
2025-12-05 20:01:49 [ayvens.es] INFO: Extracting data for car ID: 1788826
2025-12-05 20:01:49 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 08ec4e6a-234e-57c8-90ea-69d5efaac6e3 with identifier: 1788826
2025-12-05 20:01:49 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 08ec4e6a-234e-57c8-90ea-69d5efaac6e3 with scrape type 1
2025-12-05 20:01:49 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 08ec4e6a-234e-57c8-90ea-69d5efaac6e3
2025-12-05 20:01:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F862%252Fb8b9fbe5127d4379a73e1b7c72dd629c%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:49 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/862/b8b9fbe5127d4379a73e1b7c72dd629c/images/large/1.jpg not downloaded yet
2025-12-05 20:01:49 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 8d504324-e6a9-51f1-864e-25267568715d with identifier: 1788818
2025-12-05 20:01:49 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 8d504324-e6a9-51f1-864e-25267568715d with identifier: 1788818
2025-12-05 20:01:50 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'renault' containing 'renault' and model 'kangoo' containing 'kangoo'
2025-12-05 20:01:50 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 8d504324-e6a9-51f1-864e-25267568715d spider: ayvens.es
2025-12-05 20:01:50 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 8d504324-e6a9-51f1-864e-25267568715d due to rule: makes_models_not_allowed.json, error: Rejected due to make 'renault' containing 'renault' and model 'kangoo' containing 'kangoo'
{'auction_title': '',
 'c_o_2_emission_value': 121,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '8d504324-e6a9-51f1-864e-25267568715d',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'KANGOO',
 'odometer': 177638,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_8d504324-e6a9-51f1-864e-25267568715d/8d504324-e6a9-51f1-864e-25267568715d_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 313240,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/557/feddd4bc7acd402ab992dcf17c5634a2/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-07-20',
 'seats_number': 5,
 'seller_name': None,
 'title': 'RENAULT KANGOO Profesional M1-AF Blue dCi 70 kW (95 CV)',
 'transmission': 'manual',
 'trim': 'Profesional M1-AF Blue dCi 70 kW (95 CV)',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788818%27 HTTP/1.1" 200 None
2025-12-05 20:01:50 [ayvens.es] INFO: Saving data for 1788818: {'created_time': 1764964910.187257, 'last_price_update_time': 1764964910.187269, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788818') HTTP/1.1" 204 0
2025-12-05 20:01:50 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: a65f649a-65d3-5530-add8-8dd277c89621 with identifier: 1788817
2025-12-05 20:01:50 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: a65f649a-65d3-5530-add8-8dd277c89621 with identifier: 1788817
2025-12-05 20:01:50 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: a65f649a-65d3-5530-add8-8dd277c89621 with identifier: 1788817 to the API
2025-12-05 20:01:50 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:50 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:50 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/044/776362cac17349ecaedfdeb17b8964fd/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 128,
 'color': 'Rojo',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'a65f649a-65d3-5530-add8-8dd277c89621',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'AUDI',
 'model': 'A5',
 'odometer': 118301,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_a65f649a-65d3-5530-add8-8dd277c89621/a65f649a-65d3-5530-add8-8dd277c89621_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 388863,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/044/776362cac17349ecaedfdeb17b8964fd/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-03-16',
 'seats_number': 5,
 'seller_name': None,
 'title': 'AUDI A5 Advanced 35 TDI 120kW S tronic Sportback',
 'transmission': 'manual',
 'trim': 'Advanced 35 TDI 120kW S tronic Sportback',
 'vin': None,
 'year': 2021}
2025-12-05 20:01:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788817%27 HTTP/1.1" 200 None
2025-12-05 20:01:50 [ayvens.es] INFO: Saving data for 1788817: {'created_time': 1764964910.658737, 'last_price_update_time': 1764964910.658747, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788817') HTTP/1.1" 204 0
2025-12-05 20:01:50 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/535/0688f503ef384c568fd0ceff9f4364ce/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:50 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/923/e6db74e18b0f4c80b047e2d39b3cb602/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:50 [ayvens.es] INFO: Found listing with ID: 1788827
2025-12-05 20:01:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788827%27 HTTP/1.1" 200 None
2025-12-05 20:01:50 [ayvens.es] INFO: Scrape type for 1788827: 1
2025-12-05 20:01:50 [ayvens.es] INFO: Extracting data for car ID: 1788827
2025-12-05 20:01:50 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 38e6214b-6ff0-5939-917f-bf6aa7562d88 with identifier: 1788827
2025-12-05 20:01:50 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 38e6214b-6ff0-5939-917f-bf6aa7562d88 with scrape type 1
2025-12-05 20:01:50 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 38e6214b-6ff0-5939-917f-bf6aa7562d88
2025-12-05 20:01:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F141%252Fa1b63f48bb0d433ba88ecf1ac3e09eee%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:50 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/141/a1b63f48bb0d433ba88ecf1ac3e09eee/images/large/1.jpg not downloaded yet
2025-12-05 20:01:50 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/344/14af9ebacd8046d79a147eae1e0b25f4/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:50 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:50 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:50 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:50 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a71a10>
2025-12-05 20:01:50 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:50 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:50 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:01:50 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:50 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:50 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_e9c935b9-07e1-5fd7-8345-0ef36b38837f/e9c935b9-07e1-5fd7-8345-0ef36b38837f_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F682%252Fa2b71027dab949c884665b498e472086%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:51 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/682/a2b71027dab949c884665b498e472086/images/large/1.jpg saved to cache
2025-12-05 20:01:51 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item e9c935b9-07e1-5fd7-8345-0ef36b38837f
2025-12-05 20:01:51 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: e9c935b9-07e1-5fd7-8345-0ef36b38837f, skipping ID generation
2025-12-05 20:01:51 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e9c935b9-07e1-5fd7-8345-0ef36b38837f with scrape type 1
2025-12-05 20:01:51 [crawlers.middlewares.photo_download_middleware] INFO: Removing item e9c935b9-07e1-5fd7-8345-0ef36b38837f from tracking dictionaries
2025-12-05 20:01:51 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item e9c935b9-07e1-5fd7-8345-0ef36b38837f sending to next pipeline
2025-12-05 20:01:51 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: e9c935b9-07e1-5fd7-8345-0ef36b38837f, identifier: 1788820
2025-12-05 20:01:51 [ayvens.es] INFO: Found listing with ID: 1788828
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788828%27 HTTP/1.1" 200 None
2025-12-05 20:01:51 [ayvens.es] INFO: Scrape type for 1788828: 1
2025-12-05 20:01:51 [ayvens.es] INFO: Extracting data for car ID: 1788828
2025-12-05 20:01:51 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 786a21b1-5b34-5f32-90fd-eca268cd21ea with identifier: 1788828
2025-12-05 20:01:51 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 786a21b1-5b34-5f32-90fd-eca268cd21ea with scrape type 1
2025-12-05 20:01:51 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 786a21b1-5b34-5f32-90fd-eca268cd21ea
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F545%252F7d5b57494b6b461a85e79d84562066a7%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:51 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/545/7d5b57494b6b461a85e79d84562066a7/images/large/1.jpg not downloaded yet
2025-12-05 20:01:51 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: e9c935b9-07e1-5fd7-8345-0ef36b38837f with identifier: 1788820
2025-12-05 20:01:51 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: e9c935b9-07e1-5fd7-8345-0ef36b38837f with identifier: 1788820
2025-12-05 20:01:51 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: e9c935b9-07e1-5fd7-8345-0ef36b38837f with identifier: 1788820 to the API
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:51 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/682/a2b71027dab949c884665b498e472086/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 114,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '0.9L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': 'e9c935b9-07e1-5fd7-8345-0ef36b38837f',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'CLIO',
 'odometer': 58075,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_e9c935b9-07e1-5fd7-8345-0ef36b38837f/e9c935b9-07e1-5fd7-8345-0ef36b38837f_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 333416,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/682/a2b71027dab949c884665b498e472086/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-10-03',
 'seats_number': 5,
 'seller_name': None,
 'title': 'RENAULT CLIO Business TCe 55kW (75CV) -18',
 'transmission': 'manual',
 'trim': 'Business TCe 55kW (75CV) -18',
 'vin': None,
 'year': 2019}
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788820%27 HTTP/1.1" 200 None
2025-12-05 20:01:51 [ayvens.es] INFO: Saving data for 1788820: {'created_time': 1764964911.56206, 'last_price_update_time': 1764964911.562069, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788820') HTTP/1.1" 204 0
2025-12-05 20:01:51 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/873/80e49a04c57f47e98181699f4f1e7bf1/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:51 [ayvens.es] INFO: Found listing with ID: 1788829
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788829%27 HTTP/1.1" 200 None
2025-12-05 20:01:51 [ayvens.es] INFO: Scrape type for 1788829: 1
2025-12-05 20:01:51 [ayvens.es] INFO: Extracting data for car ID: 1788829
2025-12-05 20:01:51 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 83f6292a-4ce3-5c3b-9028-bc0b9751895c with identifier: 1788829
2025-12-05 20:01:51 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 83f6292a-4ce3-5c3b-9028-bc0b9751895c with scrape type 1
2025-12-05 20:01:51 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 83f6292a-4ce3-5c3b-9028-bc0b9751895c
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F209%252F5602524ef1e640a18bce22b6582795bf%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:51 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/209/5602524ef1e640a18bce22b6582795bf/images/large/1.jpg not downloaded yet
2025-12-05 20:01:51 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/336/79500f829c4f4abfb9ef79f6488c3f77/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:51 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:51 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:51 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:51 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0be97d0>
2025-12-05 20:01:51 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:51 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_0031150e-6f50-5280-840a-608ad907f154/0031150e-6f50-5280-840a-608ad907f154_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F535%252F0688f503ef384c568fd0ceff9f4364ce%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:51 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/535/0688f503ef384c568fd0ceff9f4364ce/images/large/1.jpg saved to cache
2025-12-05 20:01:51 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 0031150e-6f50-5280-840a-608ad907f154
2025-12-05 20:01:51 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 0031150e-6f50-5280-840a-608ad907f154, skipping ID generation
2025-12-05 20:01:51 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 0031150e-6f50-5280-840a-608ad907f154 with scrape type 1
2025-12-05 20:01:51 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 0031150e-6f50-5280-840a-608ad907f154 from tracking dictionaries
2025-12-05 20:01:51 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 0031150e-6f50-5280-840a-608ad907f154 sending to next pipeline
2025-12-05 20:01:52 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 0031150e-6f50-5280-840a-608ad907f154, identifier: 1788819
2025-12-05 20:01:52 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:52 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:52 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:52 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b054b490>
2025-12-05 20:01:52 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:52 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:52 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:52 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:52 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:52 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:52 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_3ae49fb1-7c59-5361-a23e-d7f95bb9ba88/3ae49fb1-7c59-5361-a23e-d7f95bb9ba88_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F923%252Fe6db74e18b0f4c80b047e2d39b3cb602%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:52 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/923/e6db74e18b0f4c80b047e2d39b3cb602/images/large/1.jpg saved to cache
2025-12-05 20:01:52 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 3ae49fb1-7c59-5361-a23e-d7f95bb9ba88
2025-12-05 20:01:52 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 3ae49fb1-7c59-5361-a23e-d7f95bb9ba88, skipping ID generation
2025-12-05 20:01:52 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 3ae49fb1-7c59-5361-a23e-d7f95bb9ba88 with scrape type 1
2025-12-05 20:01:52 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 3ae49fb1-7c59-5361-a23e-d7f95bb9ba88 from tracking dictionaries
2025-12-05 20:01:52 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 3ae49fb1-7c59-5361-a23e-d7f95bb9ba88 sending to next pipeline
2025-12-05 20:01:52 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 3ae49fb1-7c59-5361-a23e-d7f95bb9ba88, identifier: 1788821
2025-12-05 20:01:52 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:52 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:52 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:52 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0d5ded0>
2025-12-05 20:01:52 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:52 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:52 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:01:52 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:52 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:52 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:52 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_684d079c-ea8e-51e8-bcd7-87404bbefcc4/684d079c-ea8e-51e8-bcd7-87404bbefcc4_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F344%252F14af9ebacd8046d79a147eae1e0b25f4%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:52 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/344/14af9ebacd8046d79a147eae1e0b25f4/images/large/1.jpg saved to cache
2025-12-05 20:01:52 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 684d079c-ea8e-51e8-bcd7-87404bbefcc4
2025-12-05 20:01:52 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 684d079c-ea8e-51e8-bcd7-87404bbefcc4, skipping ID generation
2025-12-05 20:01:52 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 684d079c-ea8e-51e8-bcd7-87404bbefcc4 with scrape type 1
2025-12-05 20:01:52 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 684d079c-ea8e-51e8-bcd7-87404bbefcc4 from tracking dictionaries
2025-12-05 20:01:52 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 684d079c-ea8e-51e8-bcd7-87404bbefcc4 sending to next pipeline
2025-12-05 20:01:53 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 684d079c-ea8e-51e8-bcd7-87404bbefcc4, identifier: 1788822
2025-12-05 20:01:53 [ayvens.es] INFO: Found listing with ID: 1788830
2025-12-05 20:01:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788830%27 HTTP/1.1" 200 None
2025-12-05 20:01:53 [ayvens.es] INFO: Scrape type for 1788830: 1
2025-12-05 20:01:53 [ayvens.es] INFO: Extracting data for car ID: 1788830
2025-12-05 20:01:53 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: a56842d0-3f29-517d-8c5f-6c0ed2f0d6fd with identifier: 1788830
2025-12-05 20:01:53 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a56842d0-3f29-517d-8c5f-6c0ed2f0d6fd with scrape type 1
2025-12-05 20:01:53 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item a56842d0-3f29-517d-8c5f-6c0ed2f0d6fd
2025-12-05 20:01:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F916%252F82fc482fe23c4764ba58a34ce74aa6d2%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:53 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/916/82fc482fe23c4764ba58a34ce74aa6d2/images/large/1.jpg not downloaded yet
2025-12-05 20:01:53 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 0031150e-6f50-5280-840a-608ad907f154 with identifier: 1788819
2025-12-05 20:01:53 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 0031150e-6f50-5280-840a-608ad907f154 with identifier: 1788819
2025-12-05 20:01:53 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: 154056 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    154056
2025-12-05 20:01:53 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE_FOR_COUNTRY failed for item: 0031150e-6f50-5280-840a-608ad907f154 spider: ayvens.es
2025-12-05 20:01:53 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 0031150e-6f50-5280-840a-608ad907f154 due to rule: high_mileage_for_country.json, error: 154056 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    154056
{'auction_title': '',
 'c_o_2_emission_value': 133,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '0031150e-6f50-5280-840a-608ad907f154',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'ARTEON',
 'odometer': 154056,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_0031150e-6f50-5280-840a-608ad907f154/0031150e-6f50-5280-840a-608ad907f154_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 626961,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/535/0688f503ef384c568fd0ceff9f4364ce/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-12-09',
 'seats_number': 5,
 'seller_name': None,
 'title': 'VOLKSWAGEN ARTEON Elegance 2.0 TDI 110kW (150CV) DSG',
 'transmission': 'manual',
 'trim': 'Elegance 2.0 TDI 110kW (150CV) DSG',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788819%27 HTTP/1.1" 200 None
2025-12-05 20:01:53 [ayvens.es] INFO: Saving data for 1788819: {'created_time': 1764964913.368812, 'last_price_update_time': 1764964913.368824, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788819') HTTP/1.1" 204 0
2025-12-05 20:01:53 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 3ae49fb1-7c59-5361-a23e-d7f95bb9ba88 with identifier: 1788821
2025-12-05 20:01:53 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 3ae49fb1-7c59-5361-a23e-d7f95bb9ba88 with identifier: 1788821
2025-12-05 20:01:53 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'dacia' containing 'dacia' and model 'dokker' containing 'dokker'
2025-12-05 20:01:53 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 3ae49fb1-7c59-5361-a23e-d7f95bb9ba88 spider: ayvens.es
2025-12-05 20:01:53 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 3ae49fb1-7c59-5361-a23e-d7f95bb9ba88 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'dacia' containing 'dacia' and model 'dokker' containing 'dokker'
{'auction_title': '',
 'c_o_2_emission_value': 111,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '3ae49fb1-7c59-5361-a23e-d7f95bb9ba88',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'DACIA',
 'model': 'DOKKER',
 'odometer': 99241,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_3ae49fb1-7c59-5361-a23e-d7f95bb9ba88/3ae49fb1-7c59-5361-a23e-d7f95bb9ba88_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 343995,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/923/e6db74e18b0f4c80b047e2d39b3cb602/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-02-21',
 'seats_number': 5,
 'seller_name': None,
 'title': 'DACIA DOKKER Essential Blue dCi 70kW (95CV) N1',
 'transmission': 'manual',
 'trim': 'Essential Blue dCi 70kW (95CV) N1',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788821%27 HTTP/1.1" 200 None
2025-12-05 20:01:53 [ayvens.es] INFO: Saving data for 1788821: {'created_time': 1764964913.643772, 'last_price_update_time': 1764964913.643785, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788821') HTTP/1.1" 204 0
2025-12-05 20:01:53 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 684d079c-ea8e-51e8-bcd7-87404bbefcc4 with identifier: 1788822
2025-12-05 20:01:53 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 684d079c-ea8e-51e8-bcd7-87404bbefcc4 with identifier: 1788822
2025-12-05 20:01:53 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'citroën', 'model': 'c4 spacetourer', 'trim': 'puretech 96kw (130cv) s&s eat8 live', 'title': 'citroën c4 spacetourer puretech 96kw (130cv) s&s eat8 live', 'price': 0, 'odometer': 166604, 'odometer_reading_unit': 'km', 'registration_date': '2020-08-07', 'year': 2020, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.2l', 'c_o_2_emission_value': 114, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '684d079c-ea8e-51e8-bcd7-87404bbefcc4', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_684d079c-ea8e-51e8-bcd7-87404bbefcc4/684d079c-ea8e-51e8-bcd7-87404bbefcc4_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/344/14af9ebacd8046d79a147eae1e0b25f4/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 328812}], 'current_year_diff': 5, 'age_in_months': 63, 'make_model': 'citroën c4 spacetourer', 'sanitized_make': 'citron', 'odometer_in_km': 166604}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 63,
     'auction_title': '',
     'c_o_2_emission_value': 114,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.2l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': '684d079c-ea8e-51e8-bcd7-87404bbefcc4',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'citroën',
     'make_model': 'citroën c4 spacetourer',
     'model': 'c4 spacetourer',
     'odometer': 166604,
     'odometer_in_km': 166604,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_684d079c-ea8e-51e8-bcd7-87404bbefcc4/684d079c-ea8e-51e8-bcd7-87404bbefcc4_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 328812,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/344/14af9ebacd8046d79a147eae1e0b25f4/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-08-07',
     'sanitized_make': 'citron',
     'seats_number': 5,
     'seller_name': None,
     'title': 'citroën c4 spacetourer puretech 96kw (130cv) s&s eat8 live',
     'transmission': 'manual',
     'trim': 'puretech 96kw (130cv) s&s eat8 live',
     'vin': None,
     'year': 2020}
2025-12-05 20:01:53 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: PURETECH_FOR_COUNTRY failed for item: 684d079c-ea8e-51e8-bcd7-87404bbefcc4 spider: ayvens.es
2025-12-05 20:01:53 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 684d079c-ea8e-51e8-bcd7-87404bbefcc4 due to rule: puretech_for_country.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'citroën', 'model': 'c4 spacetourer', 'trim': 'puretech 96kw (130cv) s&s eat8 live', 'title': 'citroën c4 spacetourer puretech 96kw (130cv) s&s eat8 live', 'price': 0, 'odometer': 166604, 'odometer_reading_unit': 'km', 'registration_date': '2020-08-07', 'year': 2020, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.2l', 'c_o_2_emission_value': 114, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '684d079c-ea8e-51e8-bcd7-87404bbefcc4', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_684d079c-ea8e-51e8-bcd7-87404bbefcc4/684d079c-ea8e-51e8-bcd7-87404bbefcc4_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/344/14af9ebacd8046d79a147eae1e0b25f4/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 328812}], 'current_year_diff': 5, 'age_in_months': 63, 'make_model': 'citroën c4 spacetourer', 'sanitized_make': 'citron', 'odometer_in_km': 166604}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 63,
     'auction_title': '',
     'c_o_2_emission_value': 114,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.2l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': '684d079c-ea8e-51e8-bcd7-87404bbefcc4',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'citroën',
     'make_model': 'citroën c4 spacetourer',
     'model': 'c4 spacetourer',
     'odometer': 166604,
     'odometer_in_km': 166604,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_684d079c-ea8e-51e8-bcd7-87404bbefcc4/684d079c-ea8e-51e8-bcd7-87404bbefcc4_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 328812,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/344/14af9ebacd8046d79a147eae1e0b25f4/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-08-07',
     'sanitized_make': 'citron',
     'seats_number': 5,
     'seller_name': None,
     'title': 'citroën c4 spacetourer puretech 96kw (130cv) s&s eat8 live',
     'transmission': 'manual',
     'trim': 'puretech 96kw (130cv) s&s eat8 live',
     'vin': None,
     'year': 2020}
{'auction_title': '',
 'c_o_2_emission_value': 114,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.2L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '684d079c-ea8e-51e8-bcd7-87404bbefcc4',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'CITROËN',
 'model': 'C4 SPACETOURER',
 'odometer': 166604,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_684d079c-ea8e-51e8-bcd7-87404bbefcc4/684d079c-ea8e-51e8-bcd7-87404bbefcc4_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 328812,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/344/14af9ebacd8046d79a147eae1e0b25f4/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-08-07',
 'seats_number': 5,
 'seller_name': None,
 'title': 'CITROËN C4 SPACETOURER PureTech 96KW (130CV) S&S EAT8 Live',
 'transmission': 'manual',
 'trim': 'PureTech 96KW (130CV) S&S EAT8 Live',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788822%27 HTTP/1.1" 200 None
2025-12-05 20:01:53 [ayvens.es] INFO: Saving data for 1788822: {'created_time': 1764964913.914436, 'last_price_update_time': 1764964913.914447, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788822') HTTP/1.1" 204 0
2025-12-05 20:01:53 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/303/4f120c8b31484352b5f3370ab017c5b2/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:53 [ayvens.es] INFO: Found listing with ID: 1788831
2025-12-05 20:01:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788831%27 HTTP/1.1" 200 None
2025-12-05 20:01:53 [ayvens.es] INFO: Scrape type for 1788831: 1
2025-12-05 20:01:53 [ayvens.es] INFO: Extracting data for car ID: 1788831
2025-12-05 20:01:53 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 1cf2b89f-0efb-5eab-a783-f8416dadfce5 with identifier: 1788831
2025-12-05 20:01:53 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 1cf2b89f-0efb-5eab-a783-f8416dadfce5 with scrape type 1
2025-12-05 20:01:53 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 1cf2b89f-0efb-5eab-a783-f8416dadfce5
2025-12-05 20:01:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F022%252F612aaf4709d74abfbe582913c512f2d4%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:53 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/022/612aaf4709d74abfbe582913c512f2d4/images/large/1.jpg not downloaded yet
2025-12-05 20:01:53 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/862/b8b9fbe5127d4379a73e1b7c72dd629c/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:54 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:54 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:54 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:54 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0c769d0>
2025-12-05 20:01:54 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:54 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:54 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:54 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:54 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:54 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:54 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_8f37213a-fe32-561d-be2b-ba106a9c304f/8f37213a-fe32-561d-be2b-ba106a9c304f_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F873%252F80e49a04c57f47e98181699f4f1e7bf1%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:54 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/873/80e49a04c57f47e98181699f4f1e7bf1/images/large/1.jpg saved to cache
2025-12-05 20:01:54 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 8f37213a-fe32-561d-be2b-ba106a9c304f
2025-12-05 20:01:54 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 8f37213a-fe32-561d-be2b-ba106a9c304f, skipping ID generation
2025-12-05 20:01:54 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 8f37213a-fe32-561d-be2b-ba106a9c304f with scrape type 1
2025-12-05 20:01:54 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 8f37213a-fe32-561d-be2b-ba106a9c304f from tracking dictionaries
2025-12-05 20:01:54 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 8f37213a-fe32-561d-be2b-ba106a9c304f sending to next pipeline
2025-12-05 20:01:54 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 8f37213a-fe32-561d-be2b-ba106a9c304f, identifier: 1788823
2025-12-05 20:01:54 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:54 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:54 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:54 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0cd7650>
2025-12-05 20:01:54 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:54 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:54 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:01:54 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:54 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:54 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:54 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_56720714-2e01-55b3-81e9-3955f7341147/56720714-2e01-55b3-81e9-3955f7341147_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F336%252F79500f829c4f4abfb9ef79f6488c3f77%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:54 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/336/79500f829c4f4abfb9ef79f6488c3f77/images/large/1.jpg saved to cache
2025-12-05 20:01:54 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 56720714-2e01-55b3-81e9-3955f7341147
2025-12-05 20:01:54 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 56720714-2e01-55b3-81e9-3955f7341147, skipping ID generation
2025-12-05 20:01:54 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 56720714-2e01-55b3-81e9-3955f7341147 with scrape type 1
2025-12-05 20:01:54 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 56720714-2e01-55b3-81e9-3955f7341147 from tracking dictionaries
2025-12-05 20:01:54 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 56720714-2e01-55b3-81e9-3955f7341147 sending to next pipeline
2025-12-05 20:01:55 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 56720714-2e01-55b3-81e9-3955f7341147, identifier: 1788824
2025-12-05 20:01:55 [ayvens.es] INFO: Found listing with ID: 1788832
2025-12-05 20:01:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788832%27 HTTP/1.1" 200 None
2025-12-05 20:01:55 [ayvens.es] INFO: Scrape type for 1788832: 1
2025-12-05 20:01:55 [ayvens.es] INFO: Extracting data for car ID: 1788832
2025-12-05 20:01:55 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 73fc84e5-81d7-55e0-b4a2-f5526dbdc861 with identifier: 1788832
2025-12-05 20:01:55 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 73fc84e5-81d7-55e0-b4a2-f5526dbdc861 with scrape type 1
2025-12-05 20:01:55 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 73fc84e5-81d7-55e0-b4a2-f5526dbdc861
2025-12-05 20:01:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F115%252F89ac826c54524b449de120d5e99eb32a%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:55 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/115/89ac826c54524b449de120d5e99eb32a/images/large/1.jpg not downloaded yet
2025-12-05 20:01:55 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 8f37213a-fe32-561d-be2b-ba106a9c304f with identifier: 1788823
2025-12-05 20:01:55 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 8f37213a-fe32-561d-be2b-ba106a9c304f with identifier: 1788823
2025-12-05 20:01:55 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'peugeot', 'model': '308', 'trim': '5p allure puretech 130 s&s', 'title': 'peugeot 308 5p allure puretech 130 s&s', 'price': 0, 'odometer': 133836, 'odometer_reading_unit': 'km', 'registration_date': '2020-08-06', 'year': 2020, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.2l', 'c_o_2_emission_value': 102, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '8f37213a-fe32-561d-be2b-ba106a9c304f', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_8f37213a-fe32-561d-be2b-ba106a9c304f/8f37213a-fe32-561d-be2b-ba106a9c304f_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/873/80e49a04c57f47e98181699f4f1e7bf1/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 327827}], 'current_year_diff': 5, 'age_in_months': 63, 'make_model': 'peugeot 308', 'sanitized_make': 'peugeot', 'odometer_in_km': 133836}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 63,
     'auction_title': '',
     'c_o_2_emission_value': 102,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.2l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': '8f37213a-fe32-561d-be2b-ba106a9c304f',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'peugeot',
     'make_model': 'peugeot 308',
     'model': '308',
     'odometer': 133836,
     'odometer_in_km': 133836,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_8f37213a-fe32-561d-be2b-ba106a9c304f/8f37213a-fe32-561d-be2b-ba106a9c304f_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 327827,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/873/80e49a04c57f47e98181699f4f1e7bf1/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-08-06',
     'sanitized_make': 'peugeot',
     'seats_number': 5,
     'seller_name': None,
     'title': 'peugeot 308 5p allure puretech 130 s&s',
     'transmission': 'manual',
     'trim': '5p allure puretech 130 s&s',
     'vin': None,
     'year': 2020}
2025-12-05 20:01:55 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: PURETECH_FOR_COUNTRY failed for item: 8f37213a-fe32-561d-be2b-ba106a9c304f spider: ayvens.es
2025-12-05 20:01:55 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 8f37213a-fe32-561d-be2b-ba106a9c304f due to rule: puretech_for_country.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'peugeot', 'model': '308', 'trim': '5p allure puretech 130 s&s', 'title': 'peugeot 308 5p allure puretech 130 s&s', 'price': 0, 'odometer': 133836, 'odometer_reading_unit': 'km', 'registration_date': '2020-08-06', 'year': 2020, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.2l', 'c_o_2_emission_value': 102, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '8f37213a-fe32-561d-be2b-ba106a9c304f', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_8f37213a-fe32-561d-be2b-ba106a9c304f/8f37213a-fe32-561d-be2b-ba106a9c304f_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/873/80e49a04c57f47e98181699f4f1e7bf1/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 327827}], 'current_year_diff': 5, 'age_in_months': 63, 'make_model': 'peugeot 308', 'sanitized_make': 'peugeot', 'odometer_in_km': 133836}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 63,
     'auction_title': '',
     'c_o_2_emission_value': 102,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.2l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': '8f37213a-fe32-561d-be2b-ba106a9c304f',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'peugeot',
     'make_model': 'peugeot 308',
     'model': '308',
     'odometer': 133836,
     'odometer_in_km': 133836,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_8f37213a-fe32-561d-be2b-ba106a9c304f/8f37213a-fe32-561d-be2b-ba106a9c304f_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 327827,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/873/80e49a04c57f47e98181699f4f1e7bf1/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-08-06',
     'sanitized_make': 'peugeot',
     'seats_number': 5,
     'seller_name': None,
     'title': 'peugeot 308 5p allure puretech 130 s&s',
     'transmission': 'manual',
     'trim': '5p allure puretech 130 s&s',
     'vin': None,
     'year': 2020}
{'auction_title': '',
 'c_o_2_emission_value': 102,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.2L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '8f37213a-fe32-561d-be2b-ba106a9c304f',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': '308',
 'odometer': 133836,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_8f37213a-fe32-561d-be2b-ba106a9c304f/8f37213a-fe32-561d-be2b-ba106a9c304f_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 327827,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/873/80e49a04c57f47e98181699f4f1e7bf1/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-08-06',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT 308 5p Allure PureTech 130 S&S',
 'transmission': 'manual',
 'trim': '5p Allure PureTech 130 S&S',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788823%27 HTTP/1.1" 200 None
2025-12-05 20:01:55 [ayvens.es] INFO: Saving data for 1788823: {'created_time': 1764964915.337825, 'last_price_update_time': 1764964915.337839, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788823') HTTP/1.1" 204 0
2025-12-05 20:01:55 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 56720714-2e01-55b3-81e9-3955f7341147 with identifier: 1788824
2025-12-05 20:01:55 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 56720714-2e01-55b3-81e9-3955f7341147 with identifier: 1788824
2025-12-05 20:01:55 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'toyota', 'model': 'rav4', 'trim': '2.5l 220h business 4wd', 'title': 'toyota rav4 2.5l 220h business 4wd', 'price': 0, 'odometer': 213788, 'odometer_reading_unit': 'km', 'registration_date': '2020-07-27', 'year': 2020, 'fuel_type': 'hybrid', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '2.5l', 'c_o_2_emission_value': 100, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '56720714-2e01-55b3-81e9-3955f7341147', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_56720714-2e01-55b3-81e9-3955f7341147/56720714-2e01-55b3-81e9-3955f7341147_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/336/79500f829c4f4abfb9ef79f6488c3f77/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 353381}], 'current_year_diff': 5, 'age_in_months': 64, 'make_model': 'toyota rav4', 'sanitized_make': 'toyota', 'odometer_in_km': 213788}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 64,
     'auction_title': '',
     'c_o_2_emission_value': 100,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '2.5l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'hybrid',
     'id': '56720714-2e01-55b3-81e9-3955f7341147',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'toyota',
     'make_model': 'toyota rav4',
     'model': 'rav4',
     'odometer': 213788,
     'odometer_in_km': 213788,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_56720714-2e01-55b3-81e9-3955f7341147/56720714-2e01-55b3-81e9-3955f7341147_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 353381,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/336/79500f829c4f4abfb9ef79f6488c3f77/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-07-27',
     'sanitized_make': 'toyota',
     'seats_number': 5,
     'seller_name': None,
     'title': 'toyota rav4 2.5l 220h business 4wd',
     'transmission': 'manual',
     'trim': '2.5l 220h business 4wd',
     'vin': None,
     'year': 2020}
2025-12-05 20:01:55 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE failed for item: 56720714-2e01-55b3-81e9-3955f7341147 spider: ayvens.es
2025-12-05 20:01:55 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 56720714-2e01-55b3-81e9-3955f7341147 due to rule: high_mileage.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'toyota', 'model': 'rav4', 'trim': '2.5l 220h business 4wd', 'title': 'toyota rav4 2.5l 220h business 4wd', 'price': 0, 'odometer': 213788, 'odometer_reading_unit': 'km', 'registration_date': '2020-07-27', 'year': 2020, 'fuel_type': 'hybrid', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '2.5l', 'c_o_2_emission_value': 100, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '56720714-2e01-55b3-81e9-3955f7341147', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_56720714-2e01-55b3-81e9-3955f7341147/56720714-2e01-55b3-81e9-3955f7341147_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/336/79500f829c4f4abfb9ef79f6488c3f77/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 353381}], 'current_year_diff': 5, 'age_in_months': 64, 'make_model': 'toyota rav4', 'sanitized_make': 'toyota', 'odometer_in_km': 213788}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 64,
     'auction_title': '',
     'c_o_2_emission_value': 100,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '2.5l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'hybrid',
     'id': '56720714-2e01-55b3-81e9-3955f7341147',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'toyota',
     'make_model': 'toyota rav4',
     'model': 'rav4',
     'odometer': 213788,
     'odometer_in_km': 213788,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_56720714-2e01-55b3-81e9-3955f7341147/56720714-2e01-55b3-81e9-3955f7341147_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 353381,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/336/79500f829c4f4abfb9ef79f6488c3f77/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-07-27',
     'sanitized_make': 'toyota',
     'seats_number': 5,
     'seller_name': None,
     'title': 'toyota rav4 2.5l 220h business 4wd',
     'transmission': 'manual',
     'trim': '2.5l 220h business 4wd',
     'vin': None,
     'year': 2020}
{'auction_title': '',
 'c_o_2_emission_value': 100,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'hybrid',
 'id': '56720714-2e01-55b3-81e9-3955f7341147',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'TOYOTA',
 'model': 'RAV4',
 'odometer': 213788,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_56720714-2e01-55b3-81e9-3955f7341147/56720714-2e01-55b3-81e9-3955f7341147_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 353381,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/336/79500f829c4f4abfb9ef79f6488c3f77/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-07-27',
 'seats_number': 5,
 'seller_name': None,
 'title': 'TOYOTA RAV4 2.5l 220H Business 4WD',
 'transmission': 'manual',
 'trim': '2.5l 220H Business 4WD',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788824%27 HTTP/1.1" 200 None
2025-12-05 20:01:55 [ayvens.es] INFO: Saving data for 1788824: {'created_time': 1764964915.638914, 'last_price_update_time': 1764964915.638926, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788824') HTTP/1.1" 204 0
2025-12-05 20:01:55 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/141/a1b63f48bb0d433ba88ecf1ac3e09eee/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:55 [ayvens.es] INFO: Found listing with ID: 1788833
2025-12-05 20:01:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788833%27 HTTP/1.1" 200 None
2025-12-05 20:01:55 [ayvens.es] INFO: Scrape type for 1788833: 1
2025-12-05 20:01:55 [ayvens.es] INFO: Extracting data for car ID: 1788833
2025-12-05 20:01:55 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: a945b132-84ab-545d-b1b9-0e714eeab269 with identifier: 1788833
2025-12-05 20:01:55 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a945b132-84ab-545d-b1b9-0e714eeab269 with scrape type 1
2025-12-05 20:01:55 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item a945b132-84ab-545d-b1b9-0e714eeab269
2025-12-05 20:01:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F940%252F727effc282fb4c3da3b1c8c399191d91%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:55 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/940/727effc282fb4c3da3b1c8c399191d91/images/large/1.jpg not downloaded yet
2025-12-05 20:01:55 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:55 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:55 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:55 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b046a1d0>
2025-12-05 20:01:55 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:55 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:55 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:55 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:56 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:56 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:56 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_25f0daf4-d28a-5beb-84ab-9bef8b8588ea/25f0daf4-d28a-5beb-84ab-9bef8b8588ea_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:56 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:56 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F303%252F4f120c8b31484352b5f3370ab017c5b2%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:56 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/303/4f120c8b31484352b5f3370ab017c5b2/images/large/1.jpg saved to cache
2025-12-05 20:01:56 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 25f0daf4-d28a-5beb-84ab-9bef8b8588ea
2025-12-05 20:01:56 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 25f0daf4-d28a-5beb-84ab-9bef8b8588ea, skipping ID generation
2025-12-05 20:01:56 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 25f0daf4-d28a-5beb-84ab-9bef8b8588ea with scrape type 1
2025-12-05 20:01:56 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 25f0daf4-d28a-5beb-84ab-9bef8b8588ea from tracking dictionaries
2025-12-05 20:01:56 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 25f0daf4-d28a-5beb-84ab-9bef8b8588ea sending to next pipeline
2025-12-05 20:01:56 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 25f0daf4-d28a-5beb-84ab-9bef8b8588ea, identifier: 1788825
2025-12-05 20:01:56 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:56 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:56 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:56 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b04569d0>
2025-12-05 20:01:56 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:56 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:56 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:56 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:56 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:56 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:56 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_08ec4e6a-234e-57c8-90ea-69d5efaac6e3/08ec4e6a-234e-57c8-90ea-69d5efaac6e3_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:56 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:56 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F862%252Fb8b9fbe5127d4379a73e1b7c72dd629c%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:56 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/862/b8b9fbe5127d4379a73e1b7c72dd629c/images/large/1.jpg saved to cache
2025-12-05 20:01:56 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 08ec4e6a-234e-57c8-90ea-69d5efaac6e3
2025-12-05 20:01:56 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 08ec4e6a-234e-57c8-90ea-69d5efaac6e3, skipping ID generation
2025-12-05 20:01:56 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 08ec4e6a-234e-57c8-90ea-69d5efaac6e3 with scrape type 1
2025-12-05 20:01:56 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 08ec4e6a-234e-57c8-90ea-69d5efaac6e3 from tracking dictionaries
2025-12-05 20:01:56 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 08ec4e6a-234e-57c8-90ea-69d5efaac6e3 sending to next pipeline
2025-12-05 20:01:56 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 08ec4e6a-234e-57c8-90ea-69d5efaac6e3, identifier: 1788826
2025-12-05 20:01:56 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/545/7d5b57494b6b461a85e79d84562066a7/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:56 [ayvens.es] INFO: Found listing with ID: 1788834
2025-12-05 20:01:56 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:56 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788834%27 HTTP/1.1" 200 None
2025-12-05 20:01:56 [ayvens.es] INFO: Scrape type for 1788834: 1
2025-12-05 20:01:56 [ayvens.es] INFO: Extracting data for car ID: 1788834
2025-12-05 20:01:56 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 7c59aa42-335a-5289-932e-8a5022621b19 with identifier: 1788834
2025-12-05 20:01:56 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 7c59aa42-335a-5289-932e-8a5022621b19 with scrape type 1
2025-12-05 20:01:56 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 7c59aa42-335a-5289-932e-8a5022621b19
2025-12-05 20:01:56 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:56 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F245%252Fcddcd1a7a689448c83cab325e3f267ff%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:56 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/245/cddcd1a7a689448c83cab325e3f267ff/images/large/1.jpg not downloaded yet
2025-12-05 20:01:56 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 25f0daf4-d28a-5beb-84ab-9bef8b8588ea with identifier: 1788825
2025-12-05 20:01:56 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 25f0daf4-d28a-5beb-84ab-9bef8b8588ea with identifier: 1788825
2025-12-05 20:01:57 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'volkswagen', 'model': 'tiguan', 'trim': 'life 1.5 tsi 96kw (130cv)', 'title': 'volkswagen tiguan life 1.5 tsi 96kw (130cv)', 'price': 0, 'odometer': 206324, 'odometer_reading_unit': 'km', 'registration_date': '2020-09-03', 'year': 2020, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.5l', 'c_o_2_emission_value': 146, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '25f0daf4-d28a-5beb-84ab-9bef8b8588ea', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_25f0daf4-d28a-5beb-84ab-9bef8b8588ea/25f0daf4-d28a-5beb-84ab-9bef8b8588ea_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/303/4f120c8b31484352b5f3370ab017c5b2/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 348406}], 'current_year_diff': 5, 'age_in_months': 63, 'make_model': 'volkswagen tiguan', 'sanitized_make': 'volkswagen', 'odometer_in_km': 206324}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 63,
     'auction_title': '',
     'c_o_2_emission_value': 146,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.5l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': '25f0daf4-d28a-5beb-84ab-9bef8b8588ea',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'volkswagen',
     'make_model': 'volkswagen tiguan',
     'model': 'tiguan',
     'odometer': 206324,
     'odometer_in_km': 206324,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_25f0daf4-d28a-5beb-84ab-9bef8b8588ea/25f0daf4-d28a-5beb-84ab-9bef8b8588ea_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 348406,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/303/4f120c8b31484352b5f3370ab017c5b2/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-09-03',
     'sanitized_make': 'volkswagen',
     'seats_number': 5,
     'seller_name': None,
     'title': 'volkswagen tiguan life 1.5 tsi 96kw (130cv)',
     'transmission': 'manual',
     'trim': 'life 1.5 tsi 96kw (130cv)',
     'vin': None,
     'year': 2020}
2025-12-05 20:01:57 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE failed for item: 25f0daf4-d28a-5beb-84ab-9bef8b8588ea spider: ayvens.es
2025-12-05 20:01:57 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 25f0daf4-d28a-5beb-84ab-9bef8b8588ea due to rule: high_mileage.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'volkswagen', 'model': 'tiguan', 'trim': 'life 1.5 tsi 96kw (130cv)', 'title': 'volkswagen tiguan life 1.5 tsi 96kw (130cv)', 'price': 0, 'odometer': 206324, 'odometer_reading_unit': 'km', 'registration_date': '2020-09-03', 'year': 2020, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.5l', 'c_o_2_emission_value': 146, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '25f0daf4-d28a-5beb-84ab-9bef8b8588ea', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_25f0daf4-d28a-5beb-84ab-9bef8b8588ea/25f0daf4-d28a-5beb-84ab-9bef8b8588ea_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/303/4f120c8b31484352b5f3370ab017c5b2/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 348406}], 'current_year_diff': 5, 'age_in_months': 63, 'make_model': 'volkswagen tiguan', 'sanitized_make': 'volkswagen', 'odometer_in_km': 206324}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 63,
     'auction_title': '',
     'c_o_2_emission_value': 146,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.5l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': '25f0daf4-d28a-5beb-84ab-9bef8b8588ea',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'volkswagen',
     'make_model': 'volkswagen tiguan',
     'model': 'tiguan',
     'odometer': 206324,
     'odometer_in_km': 206324,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_25f0daf4-d28a-5beb-84ab-9bef8b8588ea/25f0daf4-d28a-5beb-84ab-9bef8b8588ea_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 348406,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/303/4f120c8b31484352b5f3370ab017c5b2/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-09-03',
     'sanitized_make': 'volkswagen',
     'seats_number': 5,
     'seller_name': None,
     'title': 'volkswagen tiguan life 1.5 tsi 96kw (130cv)',
     'transmission': 'manual',
     'trim': 'life 1.5 tsi 96kw (130cv)',
     'vin': None,
     'year': 2020}
{'auction_title': '',
 'c_o_2_emission_value': 146,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '25f0daf4-d28a-5beb-84ab-9bef8b8588ea',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'TIGUAN',
 'odometer': 206324,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_25f0daf4-d28a-5beb-84ab-9bef8b8588ea/25f0daf4-d28a-5beb-84ab-9bef8b8588ea_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 348406,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/303/4f120c8b31484352b5f3370ab017c5b2/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-09-03',
 'seats_number': 5,
 'seller_name': None,
 'title': 'VOLKSWAGEN TIGUAN Life 1.5 TSI 96kW (130CV)',
 'transmission': 'manual',
 'trim': 'Life 1.5 TSI 96kW (130CV)',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788825%27 HTTP/1.1" 200 None
2025-12-05 20:01:57 [ayvens.es] INFO: Saving data for 1788825: {'created_time': 1764964917.150689, 'last_price_update_time': 1764964917.150698, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788825') HTTP/1.1" 204 0
2025-12-05 20:01:57 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 08ec4e6a-234e-57c8-90ea-69d5efaac6e3 with identifier: 1788826
2025-12-05 20:01:57 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 08ec4e6a-234e-57c8-90ea-69d5efaac6e3 with identifier: 1788826
2025-12-05 20:01:57 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'seat', 'model': 'ateca', 'trim': '1.6 tdi 85kw s6s reference edition eco', 'title': 'seat ateca 1.6 tdi 85kw s6s reference edition eco', 'price': 0, 'odometer': 251094, 'odometer_reading_unit': 'km', 'registration_date': '2019-11-08', 'year': 2019, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.6l', 'c_o_2_emission_value': 120, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '08ec4e6a-234e-57c8-90ea-69d5efaac6e3', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_08ec4e6a-234e-57c8-90ea-69d5efaac6e3/08ec4e6a-234e-57c8-90ea-69d5efaac6e3_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/862/b8b9fbe5127d4379a73e1b7c72dd629c/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 337294}], 'current_year_diff': 6, 'age_in_months': 72, 'make_model': 'seat ateca', 'sanitized_make': 'seat', 'odometer_in_km': 251094}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 72,
     'auction_title': '',
     'c_o_2_emission_value': 120,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 6,
     'doors_number': 5,
     'engine': '1.6l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': '08ec4e6a-234e-57c8-90ea-69d5efaac6e3',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'seat',
     'make_model': 'seat ateca',
     'model': 'ateca',
     'odometer': 251094,
     'odometer_in_km': 251094,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_08ec4e6a-234e-57c8-90ea-69d5efaac6e3/08ec4e6a-234e-57c8-90ea-69d5efaac6e3_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 337294,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/862/b8b9fbe5127d4379a73e1b7c72dd629c/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2019-11-08',
     'sanitized_make': 'seat',
     'seats_number': 5,
     'seller_name': None,
     'title': 'seat ateca 1.6 tdi 85kw s6s reference edition eco',
     'transmission': 'manual',
     'trim': '1.6 tdi 85kw s6s reference edition eco',
     'vin': None,
     'year': 2019}
2025-12-05 20:01:57 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE failed for item: 08ec4e6a-234e-57c8-90ea-69d5efaac6e3 spider: ayvens.es
2025-12-05 20:01:57 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 08ec4e6a-234e-57c8-90ea-69d5efaac6e3 due to rule: high_mileage.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'seat', 'model': 'ateca', 'trim': '1.6 tdi 85kw s6s reference edition eco', 'title': 'seat ateca 1.6 tdi 85kw s6s reference edition eco', 'price': 0, 'odometer': 251094, 'odometer_reading_unit': 'km', 'registration_date': '2019-11-08', 'year': 2019, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.6l', 'c_o_2_emission_value': 120, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '08ec4e6a-234e-57c8-90ea-69d5efaac6e3', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_08ec4e6a-234e-57c8-90ea-69d5efaac6e3/08ec4e6a-234e-57c8-90ea-69d5efaac6e3_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/862/b8b9fbe5127d4379a73e1b7c72dd629c/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 337294}], 'current_year_diff': 6, 'age_in_months': 72, 'make_model': 'seat ateca', 'sanitized_make': 'seat', 'odometer_in_km': 251094}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 72,
     'auction_title': '',
     'c_o_2_emission_value': 120,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 6,
     'doors_number': 5,
     'engine': '1.6l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': '08ec4e6a-234e-57c8-90ea-69d5efaac6e3',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'seat',
     'make_model': 'seat ateca',
     'model': 'ateca',
     'odometer': 251094,
     'odometer_in_km': 251094,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_08ec4e6a-234e-57c8-90ea-69d5efaac6e3/08ec4e6a-234e-57c8-90ea-69d5efaac6e3_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 337294,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/862/b8b9fbe5127d4379a73e1b7c72dd629c/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2019-11-08',
     'sanitized_make': 'seat',
     'seats_number': 5,
     'seller_name': None,
     'title': 'seat ateca 1.6 tdi 85kw s6s reference edition eco',
     'transmission': 'manual',
     'trim': '1.6 tdi 85kw s6s reference edition eco',
     'vin': None,
     'year': 2019}
{'auction_title': '',
 'c_o_2_emission_value': 120,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.6L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '08ec4e6a-234e-57c8-90ea-69d5efaac6e3',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SEAT',
 'model': 'ATECA',
 'odometer': 251094,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_08ec4e6a-234e-57c8-90ea-69d5efaac6e3/08ec4e6a-234e-57c8-90ea-69d5efaac6e3_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 337294,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/862/b8b9fbe5127d4379a73e1b7c72dd629c/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-11-08',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SEAT ATECA 1.6 TDI 85kW S6S Reference Edition Eco',
 'transmission': 'manual',
 'trim': '1.6 TDI 85kW S6S Reference Edition Eco',
 'vin': None,
 'year': 2019}
2025-12-05 20:01:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788826%27 HTTP/1.1" 200 None
2025-12-05 20:01:57 [ayvens.es] INFO: Saving data for 1788826: {'created_time': 1764964917.43851, 'last_price_update_time': 1764964917.43852, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788826') HTTP/1.1" 204 0
2025-12-05 20:01:57 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/209/5602524ef1e640a18bce22b6582795bf/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:57 [ayvens.es] INFO: Found listing with ID: 1788835
2025-12-05 20:01:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788835%27 HTTP/1.1" 200 None
2025-12-05 20:01:57 [ayvens.es] INFO: Scrape type for 1788835: 1
2025-12-05 20:01:57 [ayvens.es] INFO: Extracting data for car ID: 1788835
2025-12-05 20:01:57 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: e41b479d-8efd-5b83-9d97-25809c588247 with identifier: 1788835
2025-12-05 20:01:57 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e41b479d-8efd-5b83-9d97-25809c588247 with scrape type 1
2025-12-05 20:01:57 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item e41b479d-8efd-5b83-9d97-25809c588247
2025-12-05 20:01:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F956%252Ff3a5fcbdf7b04d32a75a67268083ebc0%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:01:57 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/956/f3a5fcbdf7b04d32a75a67268083ebc0/images/large/1.jpg not downloaded yet
2025-12-05 20:01:57 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/916/82fc482fe23c4764ba58a34ce74aa6d2/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:57 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:57 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:57 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:57 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b0ce10>
2025-12-05 20:01:57 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:57 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:57 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:57 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:57 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:57 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:57 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_38e6214b-6ff0-5939-917f-bf6aa7562d88/38e6214b-6ff0-5939-917f-bf6aa7562d88_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F141%252Fa1b63f48bb0d433ba88ecf1ac3e09eee%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:57 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/141/a1b63f48bb0d433ba88ecf1ac3e09eee/images/large/1.jpg saved to cache
2025-12-05 20:01:57 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 38e6214b-6ff0-5939-917f-bf6aa7562d88
2025-12-05 20:01:57 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 38e6214b-6ff0-5939-917f-bf6aa7562d88, skipping ID generation
2025-12-05 20:01:57 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 38e6214b-6ff0-5939-917f-bf6aa7562d88 with scrape type 1
2025-12-05 20:01:57 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 38e6214b-6ff0-5939-917f-bf6aa7562d88 from tracking dictionaries
2025-12-05 20:01:57 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 38e6214b-6ff0-5939-917f-bf6aa7562d88 sending to next pipeline
2025-12-05 20:01:58 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 38e6214b-6ff0-5939-917f-bf6aa7562d88, identifier: 1788827
2025-12-05 20:01:58 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:58 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:58 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:58 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a53250>
2025-12-05 20:01:58 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:58 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:58 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:01:58 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:58 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:58 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:58 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_786a21b1-5b34-5f32-90fd-eca268cd21ea/786a21b1-5b34-5f32-90fd-eca268cd21ea_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F545%252F7d5b57494b6b461a85e79d84562066a7%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:58 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/545/7d5b57494b6b461a85e79d84562066a7/images/large/1.jpg saved to cache
2025-12-05 20:01:58 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 786a21b1-5b34-5f32-90fd-eca268cd21ea
2025-12-05 20:01:58 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 786a21b1-5b34-5f32-90fd-eca268cd21ea, skipping ID generation
2025-12-05 20:01:58 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 786a21b1-5b34-5f32-90fd-eca268cd21ea with scrape type 1
2025-12-05 20:01:58 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 786a21b1-5b34-5f32-90fd-eca268cd21ea from tracking dictionaries
2025-12-05 20:01:58 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 786a21b1-5b34-5f32-90fd-eca268cd21ea sending to next pipeline
2025-12-05 20:01:58 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 786a21b1-5b34-5f32-90fd-eca268cd21ea, identifier: 1788828
2025-12-05 20:01:58 [ayvens.es] INFO: Fetching listings with token: 3b2cc62fd26c4e29a762db3de181266b and skip: 100
2025-12-05 20:01:58 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 38e6214b-6ff0-5939-917f-bf6aa7562d88 with identifier: 1788827
2025-12-05 20:01:58 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 38e6214b-6ff0-5939-917f-bf6aa7562d88 with identifier: 1788827
2025-12-05 20:01:58 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 38e6214b-6ff0-5939-917f-bf6aa7562d88 with identifier: 1788827 to the API
2025-12-05 20:01:58 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:58 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:58 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/141/a1b63f48bb0d433ba88ecf1ac3e09eee/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 119,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '38e6214b-6ff0-5939-917f-bf6aa7562d88',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'FORD',
 'model': 'ECOSPORT',
 'odometer': 62668,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_38e6214b-6ff0-5939-917f-bf6aa7562d88/38e6214b-6ff0-5939-917f-bf6aa7562d88_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 356848,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/141/a1b63f48bb0d433ba88ecf1ac3e09eee/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-03-20',
 'seats_number': 5,
 'seller_name': None,
 'title': 'FORD ECOSPORT 1.0T EcoBoost 92kW (125CV) S&S Trend',
 'transmission': 'manual',
 'trim': '1.0T EcoBoost 92kW (125CV) S&S Trend',
 'vin': None,
 'year': 2019}
2025-12-05 20:01:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788827%27 HTTP/1.1" 200 None
2025-12-05 20:01:58 [ayvens.es] INFO: Saving data for 1788827: {'created_time': 1764964918.917473, 'last_price_update_time': 1764964918.917486, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788827') HTTP/1.1" 204 0
2025-12-05 20:01:58 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 786a21b1-5b34-5f32-90fd-eca268cd21ea with identifier: 1788828
2025-12-05 20:01:58 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 786a21b1-5b34-5f32-90fd-eca268cd21ea with identifier: 1788828
2025-12-05 20:01:59 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 786a21b1-5b34-5f32-90fd-eca268cd21ea with identifier: 1788828 to the API
2025-12-05 20:01:59 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:01:59 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:01:59 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/545/7d5b57494b6b461a85e79d84562066a7/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 109,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '786a21b1-5b34-5f32-90fd-eca268cd21ea',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': '3008',
 'odometer': 128425,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_786a21b1-5b34-5f32-90fd-eca268cd21ea/786a21b1-5b34-5f32-90fd-eca268cd21ea_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 399545,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/545/7d5b57494b6b461a85e79d84562066a7/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-01-03',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT 3008 Allure BlueHDi 96kW (130CV) S&S EAT8',
 'transmission': 'manual',
 'trim': 'Allure BlueHDi 96kW (130CV) S&S EAT8',
 'vin': None,
 'year': 2020}
2025-12-05 20:01:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788828%27 HTTP/1.1" 200 None
2025-12-05 20:01:59 [ayvens.es] INFO: Saving data for 1788828: {'created_time': 1764964919.369378, 'last_price_update_time': 1764964919.36939, 'auction_closing_time': 1765281600.0}
2025-12-05 20:01:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788828') HTTP/1.1" 204 0
2025-12-05 20:01:59 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/022/612aaf4709d74abfbe582913c512f2d4/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:01:59 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 1st time calling it.
2025-12-05 20:01:59 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:59 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:59 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:59 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b4e650>
2025-12-05 20:01:59 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:01:59 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:01:59 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:01:59 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:01:59 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:01:59 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:01:59 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_83f6292a-4ce3-5c3b-9028-bc0b9751895c/83f6292a-4ce3-5c3b-9028-bc0b9751895c_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:01:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:01:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F209%252F5602524ef1e640a18bce22b6582795bf%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:01:59 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/209/5602524ef1e640a18bce22b6582795bf/images/large/1.jpg saved to cache
2025-12-05 20:01:59 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 83f6292a-4ce3-5c3b-9028-bc0b9751895c
2025-12-05 20:01:59 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 83f6292a-4ce3-5c3b-9028-bc0b9751895c, skipping ID generation
2025-12-05 20:01:59 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 83f6292a-4ce3-5c3b-9028-bc0b9751895c with scrape type 1
2025-12-05 20:01:59 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 83f6292a-4ce3-5c3b-9028-bc0b9751895c from tracking dictionaries
2025-12-05 20:01:59 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 83f6292a-4ce3-5c3b-9028-bc0b9751895c sending to next pipeline
2025-12-05 20:01:59 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 83f6292a-4ce3-5c3b-9028-bc0b9751895c, identifier: 1788829
2025-12-05 20:01:59 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:01:59 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:01:59 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:01:59 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a51010>
2025-12-05 20:01:59 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:00 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:00 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:02:00 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:00 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:00 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:00 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_a56842d0-3f29-517d-8c5f-6c0ed2f0d6fd/a56842d0-3f29-517d-8c5f-6c0ed2f0d6fd_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F916%252F82fc482fe23c4764ba58a34ce74aa6d2%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:00 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/916/82fc482fe23c4764ba58a34ce74aa6d2/images/large/1.jpg saved to cache
2025-12-05 20:02:00 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item a56842d0-3f29-517d-8c5f-6c0ed2f0d6fd
2025-12-05 20:02:00 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: a56842d0-3f29-517d-8c5f-6c0ed2f0d6fd, skipping ID generation
2025-12-05 20:02:00 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a56842d0-3f29-517d-8c5f-6c0ed2f0d6fd with scrape type 1
2025-12-05 20:02:00 [crawlers.middlewares.photo_download_middleware] INFO: Removing item a56842d0-3f29-517d-8c5f-6c0ed2f0d6fd from tracking dictionaries
2025-12-05 20:02:00 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item a56842d0-3f29-517d-8c5f-6c0ed2f0d6fd sending to next pipeline
2025-12-05 20:02:00 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: a56842d0-3f29-517d-8c5f-6c0ed2f0d6fd, identifier: 1788830
2025-12-05 20:02:00 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 83f6292a-4ce3-5c3b-9028-bc0b9751895c with identifier: 1788829
2025-12-05 20:02:00 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 83f6292a-4ce3-5c3b-9028-bc0b9751895c with identifier: 1788829
2025-12-05 20:02:00 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: 145900 is greater than the maximum of 130000

Failed validating 'maximum' in schema['allOf'][0]['then']['properties']['odometer_in_km']:
    {'maximum': 130000, 'type': 'integer'}

On instance['odometer_in_km']:
    145900
2025-12-05 20:02:00 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE_FOR_MAKE failed for item: 83f6292a-4ce3-5c3b-9028-bc0b9751895c spider: ayvens.es
2025-12-05 20:02:00 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 83f6292a-4ce3-5c3b-9028-bc0b9751895c due to rule: high_mileage_for_make.json, error: 145900 is greater than the maximum of 130000

Failed validating 'maximum' in schema['allOf'][0]['then']['properties']['odometer_in_km']:
    {'maximum': 130000, 'type': 'integer'}

On instance['odometer_in_km']:
    145900
{'auction_title': '',
 'c_o_2_emission_value': 119,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'gas',
 'id': '83f6292a-4ce3-5c3b-9028-bc0b9751895c',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'CAPTUR',
 'odometer': 145900,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_83f6292a-4ce3-5c3b-9028-bc0b9751895c/83f6292a-4ce3-5c3b-9028-bc0b9751895c_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 356835,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/209/5602524ef1e640a18bce22b6582795bf/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-12-30',
 'seats_number': 5,
 'seller_name': None,
 'title': 'RENAULT CAPTUR Intens TCe 74kW (100CV) GLP',
 'transmission': 'manual',
 'trim': 'Intens TCe 74kW (100CV) GLP',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788829%27 HTTP/1.1" 200 None
2025-12-05 20:02:00 [ayvens.es] INFO: Saving data for 1788829: {'created_time': 1764964920.828169, 'last_price_update_time': 1764964920.828181, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788829') HTTP/1.1" 204 0
2025-12-05 20:02:00 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: a56842d0-3f29-517d-8c5f-6c0ed2f0d6fd with identifier: 1788830
2025-12-05 20:02:00 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: a56842d0-3f29-517d-8c5f-6c0ed2f0d6fd with identifier: 1788830
2025-12-05 20:02:00 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: 161857 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    161857
2025-12-05 20:02:00 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE_FOR_COUNTRY failed for item: a56842d0-3f29-517d-8c5f-6c0ed2f0d6fd spider: ayvens.es
2025-12-05 20:02:01 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: a56842d0-3f29-517d-8c5f-6c0ed2f0d6fd due to rule: high_mileage_for_country.json, error: 161857 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    161857
{'auction_title': '',
 'c_o_2_emission_value': 120,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'a56842d0-3f29-517d-8c5f-6c0ed2f0d6fd',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'MEGANE',
 'odometer': 161857,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_a56842d0-3f29-517d-8c5f-6c0ed2f0d6fd/a56842d0-3f29-517d-8c5f-6c0ed2f0d6fd_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 343342,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/916/82fc482fe23c4764ba58a34ce74aa6d2/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-05-19',
 'seats_number': 5,
 'seller_name': None,
 'title': 'RENAULT MEGANE Life Blue dCi 85 kW (115CV)',
 'transmission': 'manual',
 'trim': 'Life Blue dCi 85 kW (115CV)',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788830%27 HTTP/1.1" 200 None
2025-12-05 20:02:01 [ayvens.es] INFO: Saving data for 1788830: {'created_time': 1764964921.150909, 'last_price_update_time': 1764964921.150922, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788830') HTTP/1.1" 204 0
2025-12-05 20:02:01 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/940/727effc282fb4c3da3b1c8c399191d91/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:01 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/115/89ac826c54524b449de120d5e99eb32a/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:01 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:01 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:01 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:01 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b047a790>
2025-12-05 20:02:01 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:01 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:01 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1971
2025-12-05 20:02:01 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:01 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:01 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:01 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_1cf2b89f-0efb-5eab-a783-f8416dadfce5/1cf2b89f-0efb-5eab-a783-f8416dadfce5_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F022%252F612aaf4709d74abfbe582913c512f2d4%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:01 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/022/612aaf4709d74abfbe582913c512f2d4/images/large/1.jpg saved to cache
2025-12-05 20:02:01 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 1cf2b89f-0efb-5eab-a783-f8416dadfce5
2025-12-05 20:02:01 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 1cf2b89f-0efb-5eab-a783-f8416dadfce5, skipping ID generation
2025-12-05 20:02:01 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 1cf2b89f-0efb-5eab-a783-f8416dadfce5 with scrape type 1
2025-12-05 20:02:01 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 1cf2b89f-0efb-5eab-a783-f8416dadfce5 from tracking dictionaries
2025-12-05 20:02:01 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 1cf2b89f-0efb-5eab-a783-f8416dadfce5 sending to next pipeline
2025-12-05 20:02:01 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 1cf2b89f-0efb-5eab-a783-f8416dadfce5, identifier: 1788831
2025-12-05 20:02:01 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 1cf2b89f-0efb-5eab-a783-f8416dadfce5 with identifier: 1788831
2025-12-05 20:02:01 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 1cf2b89f-0efb-5eab-a783-f8416dadfce5 with identifier: 1788831
2025-12-05 20:02:01 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 1cf2b89f-0efb-5eab-a783-f8416dadfce5 with identifier: 1788831 to the API
2025-12-05 20:02:01 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:01 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:02 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/022/612aaf4709d74abfbe582913c512f2d4/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 147,
 'color': 'Negro',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '1cf2b89f-0efb-5eab-a783-f8416dadfce5',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'TIGUAN ALLSPACE',
 'odometer': 143509,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_1cf2b89f-0efb-5eab-a783-f8416dadfce5/1cf2b89f-0efb-5eab-a783-f8416dadfce5_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 368764,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/022/612aaf4709d74abfbe582913c512f2d4/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-01-16',
 'seats_number': 7,
 'seller_name': None,
 'title': 'VOLKSWAGEN TIGUAN ALLSPACE Sport 2.0 TDI 140kW (190CV) 4Motion DSG',
 'transmission': 'manual',
 'trim': 'Sport 2.0 TDI 140kW (190CV) 4Motion DSG',
 'vin': None,
 'year': 2020}
2025-12-05 20:02:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788831%27 HTTP/1.1" 200 None
2025-12-05 20:02:02 [ayvens.es] INFO: Saving data for 1788831: {'created_time': 1764964922.116235, 'last_price_update_time': 1764964922.116244, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788831') HTTP/1.1" 204 0
2025-12-05 20:02:02 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/245/cddcd1a7a689448c83cab325e3f267ff/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:02 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/956/f3a5fcbdf7b04d32a75a67268083ebc0/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:02 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:02 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:02 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:02 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b04b3590>
2025-12-05 20:02:02 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:02 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:02 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:02:02 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:02 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:02 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:02 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_a945b132-84ab-545d-b1b9-0e714eeab269/a945b132-84ab-545d-b1b9-0e714eeab269_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F940%252F727effc282fb4c3da3b1c8c399191d91%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:02 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/940/727effc282fb4c3da3b1c8c399191d91/images/large/1.jpg saved to cache
2025-12-05 20:02:02 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item a945b132-84ab-545d-b1b9-0e714eeab269
2025-12-05 20:02:02 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: a945b132-84ab-545d-b1b9-0e714eeab269, skipping ID generation
2025-12-05 20:02:02 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a945b132-84ab-545d-b1b9-0e714eeab269 with scrape type 1
2025-12-05 20:02:02 [crawlers.middlewares.photo_download_middleware] INFO: Removing item a945b132-84ab-545d-b1b9-0e714eeab269 from tracking dictionaries
2025-12-05 20:02:02 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item a945b132-84ab-545d-b1b9-0e714eeab269 sending to next pipeline
2025-12-05 20:02:02 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: a945b132-84ab-545d-b1b9-0e714eeab269, identifier: 1788833
2025-12-05 20:02:02 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:02 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:02 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:02 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0cd4a10>
2025-12-05 20:02:02 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:02 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:02 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:02:02 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:02 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:02 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:02 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_73fc84e5-81d7-55e0-b4a2-f5526dbdc861/73fc84e5-81d7-55e0-b4a2-f5526dbdc861_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F115%252F89ac826c54524b449de120d5e99eb32a%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:03 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/115/89ac826c54524b449de120d5e99eb32a/images/large/1.jpg saved to cache
2025-12-05 20:02:03 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 73fc84e5-81d7-55e0-b4a2-f5526dbdc861
2025-12-05 20:02:03 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 73fc84e5-81d7-55e0-b4a2-f5526dbdc861, skipping ID generation
2025-12-05 20:02:03 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 73fc84e5-81d7-55e0-b4a2-f5526dbdc861 with scrape type 1
2025-12-05 20:02:03 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 73fc84e5-81d7-55e0-b4a2-f5526dbdc861 from tracking dictionaries
2025-12-05 20:02:03 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 73fc84e5-81d7-55e0-b4a2-f5526dbdc861 sending to next pipeline
2025-12-05 20:02:03 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 73fc84e5-81d7-55e0-b4a2-f5526dbdc861, identifier: 1788832
2025-12-05 20:02:03 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: a945b132-84ab-545d-b1b9-0e714eeab269 with identifier: 1788833
2025-12-05 20:02:03 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: a945b132-84ab-545d-b1b9-0e714eeab269 with identifier: 1788833
2025-12-05 20:02:03 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: 157979 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    157979
2025-12-05 20:02:03 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE_FOR_COUNTRY failed for item: a945b132-84ab-545d-b1b9-0e714eeab269 spider: ayvens.es
2025-12-05 20:02:03 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: a945b132-84ab-545d-b1b9-0e714eeab269 due to rule: high_mileage_for_country.json, error: 157979 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    157979
{'auction_title': '',
 'c_o_2_emission_value': 116,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'a945b132-84ab-545d-b1b9-0e714eeab269',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': '2008',
 'odometer': 157979,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_a945b132-84ab-545d-b1b9-0e714eeab269/a945b132-84ab-545d-b1b9-0e714eeab269_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 329509,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/940/727effc282fb4c3da3b1c8c399191d91/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-10-28',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT 2008 Active Pack BlueHDI 81kW (110CV)',
 'transmission': 'manual',
 'trim': 'Active Pack BlueHDI 81kW (110CV)',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788833%27 HTTP/1.1" 200 None
2025-12-05 20:02:03 [ayvens.es] INFO: Saving data for 1788833: {'created_time': 1764964923.378675, 'last_price_update_time': 1764964923.378686, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788833') HTTP/1.1" 204 0
2025-12-05 20:02:03 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 73fc84e5-81d7-55e0-b4a2-f5526dbdc861 with identifier: 1788832
2025-12-05 20:02:03 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 73fc84e5-81d7-55e0-b4a2-f5526dbdc861 with identifier: 1788832
2025-12-05 20:02:03 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 73fc84e5-81d7-55e0-b4a2-f5526dbdc861 with identifier: 1788832 to the API
2025-12-05 20:02:03 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:03 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:03 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/115/89ac826c54524b449de120d5e99eb32a/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 123,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '73fc84e5-81d7-55e0-b4a2-f5526dbdc861',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'PASSAT',
 'odometer': 102371,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_73fc84e5-81d7-55e0-b4a2-f5526dbdc861/73fc84e5-81d7-55e0-b4a2-f5526dbdc861_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 624473,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/115/89ac826c54524b449de120d5e99eb32a/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-12-17',
 'seats_number': 5,
 'seller_name': None,
 'title': 'VOLKSWAGEN PASSAT Business 2.0 TDI 90kW (122CV) DSG',
 'transmission': 'manual',
 'trim': 'Business 2.0 TDI 90kW (122CV) DSG',
 'vin': None,
 'year': 2020}
2025-12-05 20:02:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788832%27 HTTP/1.1" 200 None
2025-12-05 20:02:03 [ayvens.es] INFO: Saving data for 1788832: {'created_time': 1764964923.823174, 'last_price_update_time': 1764964923.823184, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788832') HTTP/1.1" 204 0
2025-12-05 20:02:03 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:03 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:03 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:03 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b046a950>
2025-12-05 20:02:03 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:03 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:04 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:02:04 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:04 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:04 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:04 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_7c59aa42-335a-5289-932e-8a5022621b19/7c59aa42-335a-5289-932e-8a5022621b19_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F245%252Fcddcd1a7a689448c83cab325e3f267ff%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:04 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/245/cddcd1a7a689448c83cab325e3f267ff/images/large/1.jpg saved to cache
2025-12-05 20:02:04 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 7c59aa42-335a-5289-932e-8a5022621b19
2025-12-05 20:02:04 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 7c59aa42-335a-5289-932e-8a5022621b19, skipping ID generation
2025-12-05 20:02:04 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 7c59aa42-335a-5289-932e-8a5022621b19 with scrape type 1
2025-12-05 20:02:04 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 7c59aa42-335a-5289-932e-8a5022621b19 from tracking dictionaries
2025-12-05 20:02:04 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 7c59aa42-335a-5289-932e-8a5022621b19 sending to next pipeline
2025-12-05 20:02:04 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 7c59aa42-335a-5289-932e-8a5022621b19, identifier: 1788834
2025-12-05 20:02:04 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:04 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:04 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:04 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a50a50>
2025-12-05 20:02:04 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:04 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:04 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:02:04 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:04 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:04 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:04 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_e41b479d-8efd-5b83-9d97-25809c588247/e41b479d-8efd-5b83-9d97-25809c588247_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F956%252Ff3a5fcbdf7b04d32a75a67268083ebc0%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:04 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/956/f3a5fcbdf7b04d32a75a67268083ebc0/images/large/1.jpg saved to cache
2025-12-05 20:02:04 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item e41b479d-8efd-5b83-9d97-25809c588247
2025-12-05 20:02:04 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: e41b479d-8efd-5b83-9d97-25809c588247, skipping ID generation
2025-12-05 20:02:04 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e41b479d-8efd-5b83-9d97-25809c588247 with scrape type 1
2025-12-05 20:02:04 [crawlers.middlewares.photo_download_middleware] INFO: Removing item e41b479d-8efd-5b83-9d97-25809c588247 from tracking dictionaries
2025-12-05 20:02:04 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item e41b479d-8efd-5b83-9d97-25809c588247 sending to next pipeline
2025-12-05 20:02:04 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: e41b479d-8efd-5b83-9d97-25809c588247, identifier: 1788835
2025-12-05 20:02:04 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 7c59aa42-335a-5289-932e-8a5022621b19 with identifier: 1788834
2025-12-05 20:02:04 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 7c59aa42-335a-5289-932e-8a5022621b19 with identifier: 1788834
2025-12-05 20:02:04 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 7c59aa42-335a-5289-932e-8a5022621b19 with identifier: 1788834 to the API
2025-12-05 20:02:04 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:05 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/245/cddcd1a7a689448c83cab325e3f267ff/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 134,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '7c59aa42-335a-5289-932e-8a5022621b19',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'NISSAN',
 'model': 'JUKE',
 'odometer': 59288,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_7c59aa42-335a-5289-932e-8a5022621b19/7c59aa42-335a-5289-932e-8a5022621b19_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 342693,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/245/cddcd1a7a689448c83cab325e3f267ff/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-10-21',
 'seats_number': 5,
 'seller_name': None,
 'title': 'NISSAN JUKE (E) DIG-T 84 kW (114 CV) Acenta Llantas',
 'transmission': 'manual',
 'trim': '(E) DIG-T 84 kW (114 CV) Acenta Llantas',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788834%27 HTTP/1.1" 200 None
2025-12-05 20:02:05 [ayvens.es] INFO: Saving data for 1788834: {'created_time': 1764964925.219255, 'last_price_update_time': 1764964925.219266, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788834') HTTP/1.1" 204 0
2025-12-05 20:02:05 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: e41b479d-8efd-5b83-9d97-25809c588247 with identifier: 1788835
2025-12-05 20:02:05 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:02:05 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: e41b479d-8efd-5b83-9d97-25809c588247 with identifier: 1788835
2025-12-05 20:02:05 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: 168600 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    168600
2025-12-05 20:02:05 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE_FOR_COUNTRY failed for item: e41b479d-8efd-5b83-9d97-25809c588247 spider: ayvens.es
2025-12-05 20:02:05 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: e41b479d-8efd-5b83-9d97-25809c588247 due to rule: high_mileage_for_country.json, error: 168600 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    168600
{'auction_title': '',
 'c_o_2_emission_value': 159,
 'color': 'Saphirschwarz',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '3.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'e41b479d-8efd-5b83-9d97-25809c588247',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'BMW',
 'model': 'X5',
 'odometer': 168600,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_e41b479d-8efd-5b83-9d97-25809c588247/e41b479d-8efd-5b83-9d97-25809c588247_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 378659,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/956/f3a5fcbdf7b04d32a75a67268083ebc0/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-06-19',
 'seats_number': 5,
 'seller_name': None,
 'title': 'BMW X5 xDrive30d',
 'transmission': 'manual',
 'trim': 'xDrive30d',
 'vin': None,
 'year': 2020}
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788835%27 HTTP/1.1" 200 None
2025-12-05 20:02:05 [ayvens.es] INFO: Saving data for 1788835: {'created_time': 1764964925.465966, 'last_price_update_time': 1764964925.465977, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788835') HTTP/1.1" 204 0
2025-12-05 20:02:05 [scrapy.core.engine] DEBUG: Crawled (200) <POST https://api.aldcarmarket.com/graphql/saleevents> (referer: https://carmarket.ayvens.com/) ['zyte-api']
2025-12-05 20:02:05 [ayvens.es] INFO: Found listing with ID: 1788836
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788836%27 HTTP/1.1" 200 None
2025-12-05 20:02:05 [ayvens.es] INFO: Scrape type for 1788836: 1
2025-12-05 20:02:05 [ayvens.es] INFO: Extracting data for car ID: 1788836
2025-12-05 20:02:05 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 6b4ce42d-7299-5c57-bec0-89d4124b88a0 with identifier: 1788836
2025-12-05 20:02:05 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6b4ce42d-7299-5c57-bec0-89d4124b88a0 with scrape type 1
2025-12-05 20:02:05 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 6b4ce42d-7299-5c57-bec0-89d4124b88a0
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F950%252Fcdd3fb410b6e43a7a36ea01297ce5536%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:05 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/950/cdd3fb410b6e43a7a36ea01297ce5536/images/large/1.jpg not downloaded yet
2025-12-05 20:02:05 [ayvens.es] INFO: Found listing with ID: 1788837
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788837%27 HTTP/1.1" 200 None
2025-12-05 20:02:05 [ayvens.es] INFO: Scrape type for 1788837: 1
2025-12-05 20:02:05 [ayvens.es] INFO: Extracting data for car ID: 1788837
2025-12-05 20:02:05 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 6866f3c2-ce97-57d2-a8f0-d703df0bfabb with identifier: 1788837
2025-12-05 20:02:05 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6866f3c2-ce97-57d2-a8f0-d703df0bfabb with scrape type 1
2025-12-05 20:02:05 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 6866f3c2-ce97-57d2-a8f0-d703df0bfabb
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F470%252F6428e1a452ee4f6189fcb2c8535db15a%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:05 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/470/6428e1a452ee4f6189fcb2c8535db15a/images/large/1.jpg not downloaded yet
2025-12-05 20:02:05 [ayvens.es] INFO: Found listing with ID: 1788838
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788838%27 HTTP/1.1" 200 None
2025-12-05 20:02:05 [ayvens.es] INFO: Scrape type for 1788838: 1
2025-12-05 20:02:05 [ayvens.es] INFO: Extracting data for car ID: 1788838
2025-12-05 20:02:05 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: b97ae0d5-dc59-55ff-8bb4-596d01cb3fda with identifier: 1788838
2025-12-05 20:02:05 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b97ae0d5-dc59-55ff-8bb4-596d01cb3fda with scrape type 1
2025-12-05 20:02:05 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item b97ae0d5-dc59-55ff-8bb4-596d01cb3fda
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F802%252F35ad2095a5f646abbe6e8b4a802592b3%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:05 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/802/35ad2095a5f646abbe6e8b4a802592b3/images/large/1.jpg not downloaded yet
2025-12-05 20:02:05 [ayvens.es] INFO: Found listing with ID: 1788839
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788839%27 HTTP/1.1" 200 None
2025-12-05 20:02:05 [ayvens.es] INFO: Scrape type for 1788839: 1
2025-12-05 20:02:05 [ayvens.es] INFO: Extracting data for car ID: 1788839
2025-12-05 20:02:05 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: a241cc45-7664-5508-b797-1f2985dc53b2 with identifier: 1788839
2025-12-05 20:02:05 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a241cc45-7664-5508-b797-1f2985dc53b2 with scrape type 1
2025-12-05 20:02:05 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item a241cc45-7664-5508-b797-1f2985dc53b2
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F445%252F2852eab857c44f6989725b7cc2658834%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:05 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/445/2852eab857c44f6989725b7cc2658834/images/large/1.jpg not downloaded yet
2025-12-05 20:02:05 [ayvens.es] INFO: Found listing with ID: 1788840
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788840%27 HTTP/1.1" 200 None
2025-12-05 20:02:05 [ayvens.es] INFO: Scrape type for 1788840: 1
2025-12-05 20:02:05 [ayvens.es] INFO: Extracting data for car ID: 1788840
2025-12-05 20:02:05 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: b8246b90-0834-568f-b94b-afe7e9118299 with identifier: 1788840
2025-12-05 20:02:05 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b8246b90-0834-568f-b94b-afe7e9118299 with scrape type 1
2025-12-05 20:02:05 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item b8246b90-0834-568f-b94b-afe7e9118299
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F733%252Fb6acec81ab0a43eb95ca74ba692267de%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:05 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/733/b6acec81ab0a43eb95ca74ba692267de/images/large/1.jpg not downloaded yet
2025-12-05 20:02:05 [ayvens.es] INFO: Found listing with ID: 1788841
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788841%27 HTTP/1.1" 200 None
2025-12-05 20:02:05 [ayvens.es] INFO: Scrape type for 1788841: 1
2025-12-05 20:02:05 [ayvens.es] INFO: Extracting data for car ID: 1788841
2025-12-05 20:02:05 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 3a1f98d1-ae9c-50f1-892f-4dede0081e73 with identifier: 1788841
2025-12-05 20:02:05 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 3a1f98d1-ae9c-50f1-892f-4dede0081e73 with scrape type 1
2025-12-05 20:02:05 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 3a1f98d1-ae9c-50f1-892f-4dede0081e73
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F030%252F64221da2216d41658d3d68ceee9a2cbc%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:06 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/030/64221da2216d41658d3d68ceee9a2cbc/images/large/1.jpg not downloaded yet
2025-12-05 20:02:06 [ayvens.es] INFO: Found listing with ID: 1788842
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788842%27 HTTP/1.1" 200 None
2025-12-05 20:02:06 [ayvens.es] INFO: Scrape type for 1788842: 1
2025-12-05 20:02:06 [ayvens.es] INFO: Extracting data for car ID: 1788842
2025-12-05 20:02:06 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: e13cbd1a-eb43-5b61-bf64-2ddc1cfdeb85 with identifier: 1788842
2025-12-05 20:02:06 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e13cbd1a-eb43-5b61-bf64-2ddc1cfdeb85 with scrape type 1
2025-12-05 20:02:06 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item e13cbd1a-eb43-5b61-bf64-2ddc1cfdeb85
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F763%252F6aa1ea1704934a78b4c2bd59e8ce00c8%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:06 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/763/6aa1ea1704934a78b4c2bd59e8ce00c8/images/large/1.jpg not downloaded yet
2025-12-05 20:02:06 [ayvens.es] INFO: Found listing with ID: 1788843
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788843%27 HTTP/1.1" 200 None
2025-12-05 20:02:06 [ayvens.es] INFO: Scrape type for 1788843: 1
2025-12-05 20:02:06 [ayvens.es] INFO: Extracting data for car ID: 1788843
2025-12-05 20:02:06 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: a2812ac8-97cb-5c3b-b03d-886f3c42286a with identifier: 1788843
2025-12-05 20:02:06 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a2812ac8-97cb-5c3b-b03d-886f3c42286a with scrape type 1
2025-12-05 20:02:06 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item a2812ac8-97cb-5c3b-b03d-886f3c42286a
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F396%252F2f20573cd9244cc8a9a69affa4735509%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:06 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/396/2f20573cd9244cc8a9a69affa4735509/images/large/1.jpg not downloaded yet
2025-12-05 20:02:06 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/470/6428e1a452ee4f6189fcb2c8535db15a/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:06 [ayvens.es] INFO: Found listing with ID: 1788844
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788844%27 HTTP/1.1" 200 None
2025-12-05 20:02:06 [ayvens.es] INFO: Scrape type for 1788844: 1
2025-12-05 20:02:06 [ayvens.es] INFO: Extracting data for car ID: 1788844
2025-12-05 20:02:06 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 31b01c57-dcc1-5747-bd00-30b51bf5ddf9 with identifier: 1788844
2025-12-05 20:02:06 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 31b01c57-dcc1-5747-bd00-30b51bf5ddf9 with scrape type 1
2025-12-05 20:02:06 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 31b01c57-dcc1-5747-bd00-30b51bf5ddf9
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F086%252Fa6af760630274cc39e470c74fffd0f5c%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:06 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/086/a6af760630274cc39e470c74fffd0f5c/images/large/1.jpg not downloaded yet
2025-12-05 20:02:06 [ayvens.es] INFO: Found listing with ID: 1788845
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788845%27 HTTP/1.1" 200 None
2025-12-05 20:02:06 [ayvens.es] INFO: Scrape type for 1788845: 1
2025-12-05 20:02:06 [ayvens.es] INFO: Extracting data for car ID: 1788845
2025-12-05 20:02:06 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 970c6591-58a7-546c-bf2d-a3f9971ad9c4 with identifier: 1788845
2025-12-05 20:02:06 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 970c6591-58a7-546c-bf2d-a3f9971ad9c4 with scrape type 1
2025-12-05 20:02:06 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 970c6591-58a7-546c-bf2d-a3f9971ad9c4
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F635%252F0c875a59d75e4fe1bce46645419dbc3c%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:06 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/635/0c875a59d75e4fe1bce46645419dbc3c/images/large/1.jpg not downloaded yet
2025-12-05 20:02:06 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/950/cdd3fb410b6e43a7a36ea01297ce5536/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:06 [ayvens.es] INFO: Found listing with ID: 1788846
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788846%27 HTTP/1.1" 200 None
2025-12-05 20:02:06 [ayvens.es] INFO: Scrape type for 1788846: 1
2025-12-05 20:02:06 [ayvens.es] INFO: Extracting data for car ID: 1788846
2025-12-05 20:02:06 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: ad487e32-9ed9-51c0-b6f3-120ad68435c9 with identifier: 1788846
2025-12-05 20:02:06 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ad487e32-9ed9-51c0-b6f3-120ad68435c9 with scrape type 1
2025-12-05 20:02:06 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item ad487e32-9ed9-51c0-b6f3-120ad68435c9
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F796%252Fe87f095dfbd9408094b966298820d438%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:06 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/796/e87f095dfbd9408094b966298820d438/images/large/1.jpg not downloaded yet
2025-12-05 20:02:06 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/445/2852eab857c44f6989725b7cc2658834/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:06 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/802/35ad2095a5f646abbe6e8b4a802592b3/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:06 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:06 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:06 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:06 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b04577d0>
2025-12-05 20:02:06 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:06 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_6866f3c2-ce97-57d2-a8f0-d703df0bfabb/6866f3c2-ce97-57d2-a8f0-d703df0bfabb_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F470%252F6428e1a452ee4f6189fcb2c8535db15a%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:06 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/470/6428e1a452ee4f6189fcb2c8535db15a/images/large/1.jpg saved to cache
2025-12-05 20:02:06 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 6866f3c2-ce97-57d2-a8f0-d703df0bfabb
2025-12-05 20:02:06 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 6866f3c2-ce97-57d2-a8f0-d703df0bfabb, skipping ID generation
2025-12-05 20:02:06 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6866f3c2-ce97-57d2-a8f0-d703df0bfabb with scrape type 1
2025-12-05 20:02:06 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 6866f3c2-ce97-57d2-a8f0-d703df0bfabb from tracking dictionaries
2025-12-05 20:02:06 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 6866f3c2-ce97-57d2-a8f0-d703df0bfabb sending to next pipeline
2025-12-05 20:02:06 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 6866f3c2-ce97-57d2-a8f0-d703df0bfabb, identifier: 1788837
2025-12-05 20:02:06 [ayvens.es] INFO: Found listing with ID: 1788847
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788847%27 HTTP/1.1" 200 None
2025-12-05 20:02:06 [ayvens.es] INFO: Scrape type for 1788847: 1
2025-12-05 20:02:06 [ayvens.es] INFO: Extracting data for car ID: 1788847
2025-12-05 20:02:06 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 120a4e5e-f749-5270-a76d-0ddc77285650 with identifier: 1788847
2025-12-05 20:02:06 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 120a4e5e-f749-5270-a76d-0ddc77285650 with scrape type 1
2025-12-05 20:02:06 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 120a4e5e-f749-5270-a76d-0ddc77285650
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F110%252Fd85cda5b851245a58e04a8a144625151%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:06 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/110/d85cda5b851245a58e04a8a144625151/images/large/1.jpg not downloaded yet
2025-12-05 20:02:06 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 6866f3c2-ce97-57d2-a8f0-d703df0bfabb with identifier: 1788837
2025-12-05 20:02:06 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 6866f3c2-ce97-57d2-a8f0-d703df0bfabb with identifier: 1788837
2025-12-05 20:02:06 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 6866f3c2-ce97-57d2-a8f0-d703df0bfabb with identifier: 1788837 to the API
2025-12-05 20:02:06 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:07 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:07 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/470/6428e1a452ee4f6189fcb2c8535db15a/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 125,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '6866f3c2-ce97-57d2-a8f0-d703df0bfabb',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SKODA',
 'model': 'KAMIQ',
 'odometer': 28498,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_6866f3c2-ce97-57d2-a8f0-d703df0bfabb/6866f3c2-ce97-57d2-a8f0-d703df0bfabb_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 343198,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/470/6428e1a452ee4f6189fcb2c8535db15a/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2023-12-22',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SKODA KAMIQ 1.0 TSI 85kW (115CV) SELECTION',
 'transmission': 'manual',
 'trim': '1.0 TSI 85kW (115CV) SELECTION',
 'vin': None,
 'year': 2023}
2025-12-05 20:02:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788837%27 HTTP/1.1" 200 None
2025-12-05 20:02:07 [ayvens.es] INFO: Saving data for 1788837: {'created_time': 1764964927.257001, 'last_price_update_time': 1764964927.257015, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788837') HTTP/1.1" 204 0
2025-12-05 20:02:07 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/733/b6acec81ab0a43eb95ca74ba692267de/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:07 [ayvens.es] INFO: Found listing with ID: 1788848
2025-12-05 20:02:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788848%27 HTTP/1.1" 200 None
2025-12-05 20:02:07 [ayvens.es] INFO: Scrape type for 1788848: 1
2025-12-05 20:02:07 [ayvens.es] INFO: Extracting data for car ID: 1788848
2025-12-05 20:02:07 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: c3b6bbbb-79bb-5038-ae4e-40d22d319033 with identifier: 1788848
2025-12-05 20:02:07 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item c3b6bbbb-79bb-5038-ae4e-40d22d319033 with scrape type 1
2025-12-05 20:02:07 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item c3b6bbbb-79bb-5038-ae4e-40d22d319033
2025-12-05 20:02:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F582%252F1926aa357d714e15ba1791530bd7c090%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:07 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/582/1926aa357d714e15ba1791530bd7c090/images/large/1.jpg not downloaded yet
2025-12-05 20:02:07 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:07 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:07 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:07 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b916d0>
2025-12-05 20:02:07 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:07 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:07 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1969
2025-12-05 20:02:07 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:07 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:07 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:07 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_6b4ce42d-7299-5c57-bec0-89d4124b88a0/6b4ce42d-7299-5c57-bec0-89d4124b88a0_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F950%252Fcdd3fb410b6e43a7a36ea01297ce5536%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:07 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/950/cdd3fb410b6e43a7a36ea01297ce5536/images/large/1.jpg saved to cache
2025-12-05 20:02:07 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 6b4ce42d-7299-5c57-bec0-89d4124b88a0
2025-12-05 20:02:07 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 6b4ce42d-7299-5c57-bec0-89d4124b88a0, skipping ID generation
2025-12-05 20:02:07 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6b4ce42d-7299-5c57-bec0-89d4124b88a0 with scrape type 1
2025-12-05 20:02:07 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 6b4ce42d-7299-5c57-bec0-89d4124b88a0 from tracking dictionaries
2025-12-05 20:02:07 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 6b4ce42d-7299-5c57-bec0-89d4124b88a0 sending to next pipeline
2025-12-05 20:02:07 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 6b4ce42d-7299-5c57-bec0-89d4124b88a0, identifier: 1788836
2025-12-05 20:02:07 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:07 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:07 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:07 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0686310>
2025-12-05 20:02:07 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:07 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:08 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:02:08 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:08 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:08 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:08 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_a241cc45-7664-5508-b797-1f2985dc53b2/a241cc45-7664-5508-b797-1f2985dc53b2_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:08 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:08 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F445%252F2852eab857c44f6989725b7cc2658834%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:08 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/445/2852eab857c44f6989725b7cc2658834/images/large/1.jpg saved to cache
2025-12-05 20:02:08 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item a241cc45-7664-5508-b797-1f2985dc53b2
2025-12-05 20:02:08 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: a241cc45-7664-5508-b797-1f2985dc53b2, skipping ID generation
2025-12-05 20:02:08 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a241cc45-7664-5508-b797-1f2985dc53b2 with scrape type 1
2025-12-05 20:02:08 [crawlers.middlewares.photo_download_middleware] INFO: Removing item a241cc45-7664-5508-b797-1f2985dc53b2 from tracking dictionaries
2025-12-05 20:02:08 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item a241cc45-7664-5508-b797-1f2985dc53b2 sending to next pipeline
2025-12-05 20:02:08 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: a241cc45-7664-5508-b797-1f2985dc53b2, identifier: 1788839
2025-12-05 20:02:08 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:08 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:08 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:08 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a70e90>
2025-12-05 20:02:08 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:08 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:08 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1969
2025-12-05 20:02:08 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:08 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:08 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:08 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_b97ae0d5-dc59-55ff-8bb4-596d01cb3fda/b97ae0d5-dc59-55ff-8bb4-596d01cb3fda_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:08 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:08 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F802%252F35ad2095a5f646abbe6e8b4a802592b3%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:08 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/802/35ad2095a5f646abbe6e8b4a802592b3/images/large/1.jpg saved to cache
2025-12-05 20:02:08 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item b97ae0d5-dc59-55ff-8bb4-596d01cb3fda
2025-12-05 20:02:08 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: b97ae0d5-dc59-55ff-8bb4-596d01cb3fda, skipping ID generation
2025-12-05 20:02:08 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b97ae0d5-dc59-55ff-8bb4-596d01cb3fda with scrape type 1
2025-12-05 20:02:08 [crawlers.middlewares.photo_download_middleware] INFO: Removing item b97ae0d5-dc59-55ff-8bb4-596d01cb3fda from tracking dictionaries
2025-12-05 20:02:08 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item b97ae0d5-dc59-55ff-8bb4-596d01cb3fda sending to next pipeline
2025-12-05 20:02:08 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: b97ae0d5-dc59-55ff-8bb4-596d01cb3fda, identifier: 1788838
2025-12-05 20:02:08 [scrapy.extensions.logstats] INFO: Crawled 4 pages (at 0 pages/min), scraped 58 items (at 36 items/min)
2025-12-05 20:02:08 [ayvens.es] INFO: Found listing with ID: 1788849
2025-12-05 20:02:08 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:08 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788849%27 HTTP/1.1" 200 None
2025-12-05 20:02:08 [ayvens.es] INFO: Scrape type for 1788849: 1
2025-12-05 20:02:08 [ayvens.es] INFO: Extracting data for car ID: 1788849
2025-12-05 20:02:08 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: c4e9a70a-cdd6-55dd-a511-bca4aaee7393 with identifier: 1788849
2025-12-05 20:02:08 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item c4e9a70a-cdd6-55dd-a511-bca4aaee7393 with scrape type 1
2025-12-05 20:02:08 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item c4e9a70a-cdd6-55dd-a511-bca4aaee7393
2025-12-05 20:02:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F868%252F533e3ae516f04cbfa41f3a4374173671%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:09 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/868/533e3ae516f04cbfa41f3a4374173671/images/large/1.jpg not downloaded yet
2025-12-05 20:02:09 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 6b4ce42d-7299-5c57-bec0-89d4124b88a0 with identifier: 1788836
2025-12-05 20:02:09 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 6b4ce42d-7299-5c57-bec0-89d4124b88a0 with identifier: 1788836
2025-12-05 20:02:09 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: 151205 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    151205
2025-12-05 20:02:09 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE_FOR_COUNTRY failed for item: 6b4ce42d-7299-5c57-bec0-89d4124b88a0 spider: ayvens.es
2025-12-05 20:02:09 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 6b4ce42d-7299-5c57-bec0-89d4124b88a0 due to rule: high_mileage_for_country.json, error: 151205 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    151205
{'auction_title': '',
 'c_o_2_emission_value': 96,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '6b4ce42d-7299-5c57-bec0-89d4124b88a0',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'CITROËN',
 'model': 'C3',
 'odometer': 151205,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_6b4ce42d-7299-5c57-bec0-89d4124b88a0/6b4ce42d-7299-5c57-bec0-89d4124b88a0_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 339628,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/950/cdd3fb410b6e43a7a36ea01297ce5536/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-12-10',
 'seats_number': 5,
 'seller_name': None,
 'title': 'CITROËN C3 BlueHDi 73KW (100CV) S&S FEEL',
 'transmission': 'manual',
 'trim': 'BlueHDi 73KW (100CV) S&S FEEL',
 'vin': None,
 'year': 2019}
2025-12-05 20:02:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788836%27 HTTP/1.1" 200 None
2025-12-05 20:02:09 [ayvens.es] INFO: Saving data for 1788836: {'created_time': 1764964929.283624, 'last_price_update_time': 1764964929.283638, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788836') HTTP/1.1" 204 0
2025-12-05 20:02:09 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: a241cc45-7664-5508-b797-1f2985dc53b2 with identifier: 1788839
2025-12-05 20:02:09 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: a241cc45-7664-5508-b797-1f2985dc53b2 with identifier: 1788839
2025-12-05 20:02:09 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: a241cc45-7664-5508-b797-1f2985dc53b2 with identifier: 1788839 to the API
2025-12-05 20:02:09 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:09 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:09 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/445/2852eab857c44f6989725b7cc2658834/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 121,
 'color': 'Azul',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.9L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'a241cc45-7664-5508-b797-1f2985dc53b2',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'MERCEDES-BENZ',
 'model': 'GLA',
 'odometer': 55087,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_a241cc45-7664-5508-b797-1f2985dc53b2/a241cc45-7664-5508-b797-1f2985dc53b2_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 521350,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/445/2852eab857c44f6989725b7cc2658834/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-12-28',
 'seats_number': 5,
 'seller_name': None,
 'title': 'MERCEDES-BENZ GLA GLA 200 D',
 'transmission': 'manual',
 'trim': 'GLA 200 D',
 'vin': None,
 'year': 2020}
2025-12-05 20:02:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788839%27 HTTP/1.1" 200 None
2025-12-05 20:02:09 [ayvens.es] INFO: Saving data for 1788839: {'created_time': 1764964929.624657, 'last_price_update_time': 1764964929.624668, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788839') HTTP/1.1" 204 0
2025-12-05 20:02:09 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: b97ae0d5-dc59-55ff-8bb4-596d01cb3fda with identifier: 1788838
2025-12-05 20:02:09 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:02:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:02:09 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: b97ae0d5-dc59-55ff-8bb4-596d01cb3fda with identifier: 1788838
2025-12-05 20:02:09 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: b97ae0d5-dc59-55ff-8bb4-596d01cb3fda with identifier: 1788838 to the API
2025-12-05 20:02:09 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:09 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:09 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/802/35ad2095a5f646abbe6e8b4a802592b3/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 205,
 'color': 'Alpinweib',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '3.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': 'b97ae0d5-dc59-55ff-8bb4-596d01cb3fda',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'BMW',
 'model': 'X5',
 'odometer': 82167,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_b97ae0d5-dc59-55ff-8bb4-596d01cb3fda/b97ae0d5-dc59-55ff-8bb4-596d01cb3fda_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 373340,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/802/35ad2095a5f646abbe6e8b4a802592b3/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-11-15',
 'seats_number': 5,
 'seller_name': None,
 'title': 'BMW X5 xDrive40i',
 'transmission': 'manual',
 'trim': 'xDrive40i',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788838%27 HTTP/1.1" 200 None
2025-12-05 20:02:09 [ayvens.es] INFO: Saving data for 1788838: {'created_time': 1764964929.99538, 'last_price_update_time': 1764964929.995392, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788838') HTTP/1.1" 204 0
2025-12-05 20:02:10 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/396/2f20573cd9244cc8a9a69affa4735509/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:10 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/763/6aa1ea1704934a78b4c2bd59e8ce00c8/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:10 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/030/64221da2216d41658d3d68ceee9a2cbc/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:10 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/086/a6af760630274cc39e470c74fffd0f5c/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:10 [ayvens.es] INFO: Found listing with ID: 1788850
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788850%27 HTTP/1.1" 200 None
2025-12-05 20:02:10 [ayvens.es] INFO: Scrape type for 1788850: 1
2025-12-05 20:02:10 [ayvens.es] INFO: Extracting data for car ID: 1788850
2025-12-05 20:02:10 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: bdecba64-e44e-5683-8a9b-cbddfaf80ff7 with identifier: 1788850
2025-12-05 20:02:10 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item bdecba64-e44e-5683-8a9b-cbddfaf80ff7 with scrape type 1
2025-12-05 20:02:10 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item bdecba64-e44e-5683-8a9b-cbddfaf80ff7
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F524%252F8453b2cb5d6c43e588c3528f6ee67ad9%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:10 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/524/8453b2cb5d6c43e588c3528f6ee67ad9/images/large/1.jpg not downloaded yet
2025-12-05 20:02:10 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/635/0c875a59d75e4fe1bce46645419dbc3c/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:10 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:10 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:10 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:10 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b05e0ad0>
2025-12-05 20:02:10 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:10 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_b8246b90-0834-568f-b94b-afe7e9118299/b8246b90-0834-568f-b94b-afe7e9118299_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F733%252Fb6acec81ab0a43eb95ca74ba692267de%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:10 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/733/b6acec81ab0a43eb95ca74ba692267de/images/large/1.jpg saved to cache
2025-12-05 20:02:10 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item b8246b90-0834-568f-b94b-afe7e9118299
2025-12-05 20:02:10 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: b8246b90-0834-568f-b94b-afe7e9118299, skipping ID generation
2025-12-05 20:02:10 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b8246b90-0834-568f-b94b-afe7e9118299 with scrape type 1
2025-12-05 20:02:10 [crawlers.middlewares.photo_download_middleware] INFO: Removing item b8246b90-0834-568f-b94b-afe7e9118299 from tracking dictionaries
2025-12-05 20:02:10 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item b8246b90-0834-568f-b94b-afe7e9118299 sending to next pipeline
2025-12-05 20:02:10 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: b8246b90-0834-568f-b94b-afe7e9118299, identifier: 1788840
2025-12-05 20:02:10 [ayvens.es] INFO: Found listing with ID: 1788851
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788851%27 HTTP/1.1" 200 None
2025-12-05 20:02:10 [ayvens.es] INFO: Scrape type for 1788851: 1
2025-12-05 20:02:10 [ayvens.es] INFO: Extracting data for car ID: 1788851
2025-12-05 20:02:10 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 919f3336-9283-5ec1-b889-5db63c0c2967 with identifier: 1788851
2025-12-05 20:02:10 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 919f3336-9283-5ec1-b889-5db63c0c2967 with scrape type 1
2025-12-05 20:02:10 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 919f3336-9283-5ec1-b889-5db63c0c2967
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F249%252F18a8de782fbf4710968a365afc256761%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:10 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/249/18a8de782fbf4710968a365afc256761/images/large/1.jpg not downloaded yet
2025-12-05 20:02:10 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: b8246b90-0834-568f-b94b-afe7e9118299 with identifier: 1788840
2025-12-05 20:02:10 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: b8246b90-0834-568f-b94b-afe7e9118299 with identifier: 1788840
2025-12-05 20:02:10 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: b8246b90-0834-568f-b94b-afe7e9118299 with identifier: 1788840 to the API
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:10 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/733/b6acec81ab0a43eb95ca74ba692267de/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 120,
 'color': 'Negro',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'b8246b90-0834-568f-b94b-afe7e9118299',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': '508',
 'odometer': 55679,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_b8246b90-0834-568f-b94b-afe7e9118299/b8246b90-0834-568f-b94b-afe7e9118299_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 412146,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/733/b6acec81ab0a43eb95ca74ba692267de/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2022-06-02',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT 508 5P Active Pack BlueHDi 130 S&S EAT8',
 'transmission': 'manual',
 'trim': '5P Active Pack BlueHDi 130 S&S EAT8',
 'vin': None,
 'year': 2022}
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788840%27 HTTP/1.1" 200 None
2025-12-05 20:02:10 [ayvens.es] INFO: Saving data for 1788840: {'created_time': 1764964930.905558, 'last_price_update_time': 1764964930.905569, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788840') HTTP/1.1" 204 0
2025-12-05 20:02:10 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/796/e87f095dfbd9408094b966298820d438/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:10 [ayvens.es] INFO: Found listing with ID: 1788852
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788852%27 HTTP/1.1" 200 None
2025-12-05 20:02:10 [ayvens.es] INFO: Scrape type for 1788852: 1
2025-12-05 20:02:10 [ayvens.es] INFO: Extracting data for car ID: 1788852
2025-12-05 20:02:10 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 2f7443d0-fa37-57c5-a327-4a7cf275dee2 with identifier: 1788852
2025-12-05 20:02:10 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 2f7443d0-fa37-57c5-a327-4a7cf275dee2 with scrape type 1
2025-12-05 20:02:10 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 2f7443d0-fa37-57c5-a327-4a7cf275dee2
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F981%252F738216625f66483b9377f1c23b8a7790%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:10 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/981/738216625f66483b9377f1c23b8a7790/images/large/1.jpg not downloaded yet
2025-12-05 20:02:10 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/110/d85cda5b851245a58e04a8a144625151/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:10 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:10 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:10 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:10 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0542a90>
2025-12-05 20:02:10 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:10 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:11 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:02:11 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:11 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:11 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:11 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_a2812ac8-97cb-5c3b-b03d-886f3c42286a/a2812ac8-97cb-5c3b-b03d-886f3c42286a_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F396%252F2f20573cd9244cc8a9a69affa4735509%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:11 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/396/2f20573cd9244cc8a9a69affa4735509/images/large/1.jpg saved to cache
2025-12-05 20:02:11 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item a2812ac8-97cb-5c3b-b03d-886f3c42286a
2025-12-05 20:02:11 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: a2812ac8-97cb-5c3b-b03d-886f3c42286a, skipping ID generation
2025-12-05 20:02:11 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a2812ac8-97cb-5c3b-b03d-886f3c42286a with scrape type 1
2025-12-05 20:02:11 [crawlers.middlewares.photo_download_middleware] INFO: Removing item a2812ac8-97cb-5c3b-b03d-886f3c42286a from tracking dictionaries
2025-12-05 20:02:11 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item a2812ac8-97cb-5c3b-b03d-886f3c42286a sending to next pipeline
2025-12-05 20:02:11 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: a2812ac8-97cb-5c3b-b03d-886f3c42286a, identifier: 1788843
2025-12-05 20:02:11 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:11 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:11 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:11 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b79a90>
2025-12-05 20:02:11 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:11 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:11 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:02:11 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:11 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:11 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:11 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_e13cbd1a-eb43-5b61-bf64-2ddc1cfdeb85/e13cbd1a-eb43-5b61-bf64-2ddc1cfdeb85_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F763%252F6aa1ea1704934a78b4c2bd59e8ce00c8%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:11 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/763/6aa1ea1704934a78b4c2bd59e8ce00c8/images/large/1.jpg saved to cache
2025-12-05 20:02:11 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item e13cbd1a-eb43-5b61-bf64-2ddc1cfdeb85
2025-12-05 20:02:11 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: e13cbd1a-eb43-5b61-bf64-2ddc1cfdeb85, skipping ID generation
2025-12-05 20:02:11 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e13cbd1a-eb43-5b61-bf64-2ddc1cfdeb85 with scrape type 1
2025-12-05 20:02:11 [crawlers.middlewares.photo_download_middleware] INFO: Removing item e13cbd1a-eb43-5b61-bf64-2ddc1cfdeb85 from tracking dictionaries
2025-12-05 20:02:11 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item e13cbd1a-eb43-5b61-bf64-2ddc1cfdeb85 sending to next pipeline
2025-12-05 20:02:11 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: e13cbd1a-eb43-5b61-bf64-2ddc1cfdeb85, identifier: 1788842
2025-12-05 20:02:11 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:11 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:11 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:11 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a73cd0>
2025-12-05 20:02:11 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:11 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:12 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:02:12 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:12 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:12 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:12 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_3a1f98d1-ae9c-50f1-892f-4dede0081e73/3a1f98d1-ae9c-50f1-892f-4dede0081e73_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F030%252F64221da2216d41658d3d68ceee9a2cbc%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:12 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/030/64221da2216d41658d3d68ceee9a2cbc/images/large/1.jpg saved to cache
2025-12-05 20:02:12 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 3a1f98d1-ae9c-50f1-892f-4dede0081e73
2025-12-05 20:02:12 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 3a1f98d1-ae9c-50f1-892f-4dede0081e73, skipping ID generation
2025-12-05 20:02:12 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 3a1f98d1-ae9c-50f1-892f-4dede0081e73 with scrape type 1
2025-12-05 20:02:12 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 3a1f98d1-ae9c-50f1-892f-4dede0081e73 from tracking dictionaries
2025-12-05 20:02:12 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 3a1f98d1-ae9c-50f1-892f-4dede0081e73 sending to next pipeline
2025-12-05 20:02:12 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 3a1f98d1-ae9c-50f1-892f-4dede0081e73, identifier: 1788841
2025-12-05 20:02:12 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:12 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:12 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:12 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0485910>
2025-12-05 20:02:12 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:12 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:12 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:02:12 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:12 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:12 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:12 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_31b01c57-dcc1-5747-bd00-30b51bf5ddf9/31b01c57-dcc1-5747-bd00-30b51bf5ddf9_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F086%252Fa6af760630274cc39e470c74fffd0f5c%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:12 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/086/a6af760630274cc39e470c74fffd0f5c/images/large/1.jpg saved to cache
2025-12-05 20:02:12 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 31b01c57-dcc1-5747-bd00-30b51bf5ddf9
2025-12-05 20:02:12 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 31b01c57-dcc1-5747-bd00-30b51bf5ddf9, skipping ID generation
2025-12-05 20:02:12 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 31b01c57-dcc1-5747-bd00-30b51bf5ddf9 with scrape type 1
2025-12-05 20:02:12 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 31b01c57-dcc1-5747-bd00-30b51bf5ddf9 from tracking dictionaries
2025-12-05 20:02:12 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 31b01c57-dcc1-5747-bd00-30b51bf5ddf9 sending to next pipeline
2025-12-05 20:02:12 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 31b01c57-dcc1-5747-bd00-30b51bf5ddf9, identifier: 1788844
2025-12-05 20:02:12 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:12 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:12 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:12 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b04a9090>
2025-12-05 20:02:12 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:12 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:13 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:02:13 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:13 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:13 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:13 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_970c6591-58a7-546c-bf2d-a3f9971ad9c4/970c6591-58a7-546c-bf2d-a3f9971ad9c4_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F635%252F0c875a59d75e4fe1bce46645419dbc3c%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:13 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/635/0c875a59d75e4fe1bce46645419dbc3c/images/large/1.jpg saved to cache
2025-12-05 20:02:13 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 970c6591-58a7-546c-bf2d-a3f9971ad9c4
2025-12-05 20:02:13 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 970c6591-58a7-546c-bf2d-a3f9971ad9c4, skipping ID generation
2025-12-05 20:02:13 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 970c6591-58a7-546c-bf2d-a3f9971ad9c4 with scrape type 1
2025-12-05 20:02:13 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 970c6591-58a7-546c-bf2d-a3f9971ad9c4 from tracking dictionaries
2025-12-05 20:02:13 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 970c6591-58a7-546c-bf2d-a3f9971ad9c4 sending to next pipeline
2025-12-05 20:02:13 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 970c6591-58a7-546c-bf2d-a3f9971ad9c4, identifier: 1788845
2025-12-05 20:02:13 [ayvens.es] INFO: Found listing with ID: 1788853
2025-12-05 20:02:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788853%27 HTTP/1.1" 200 None
2025-12-05 20:02:13 [ayvens.es] INFO: Scrape type for 1788853: 1
2025-12-05 20:02:13 [ayvens.es] INFO: Extracting data for car ID: 1788853
2025-12-05 20:02:13 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: d694b55c-c504-5406-93ef-7b8370add0fd with identifier: 1788853
2025-12-05 20:02:13 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item d694b55c-c504-5406-93ef-7b8370add0fd with scrape type 1
2025-12-05 20:02:13 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item d694b55c-c504-5406-93ef-7b8370add0fd
2025-12-05 20:02:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F282%252F93a166eb803a4694aa407909176a4f38%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:13 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/282/93a166eb803a4694aa407909176a4f38/images/large/1.jpg not downloaded yet
2025-12-05 20:02:13 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: a2812ac8-97cb-5c3b-b03d-886f3c42286a with identifier: 1788843
2025-12-05 20:02:13 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: a2812ac8-97cb-5c3b-b03d-886f3c42286a with identifier: 1788843
2025-12-05 20:02:13 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: a2812ac8-97cb-5c3b-b03d-886f3c42286a with identifier: 1788843 to the API
2025-12-05 20:02:13 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:13 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:13 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/396/2f20573cd9244cc8a9a69affa4735509/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 102,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.6L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'a2812ac8-97cb-5c3b-b03d-886f3c42286a',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'TALISMAN',
 'odometer': 127200,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_a2812ac8-97cb-5c3b-b03d-886f3c42286a/a2812ac8-97cb-5c3b-b03d-886f3c42286a_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 340630,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/396/2f20573cd9244cc8a9a69affa4735509/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2018-07-10',
 'seats_number': 5,
 'seller_name': None,
 'title': 'RENAULT TALISMAN Zen Energy dCi 96kW (130CV)',
 'transmission': 'manual',
 'trim': 'Zen Energy dCi 96kW (130CV)',
 'vin': None,
 'year': 2018}
2025-12-05 20:02:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788843%27 HTTP/1.1" 200 None
2025-12-05 20:02:13 [ayvens.es] INFO: Saving data for 1788843: {'created_time': 1764964933.783098, 'last_price_update_time': 1764964933.78311, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788843') HTTP/1.1" 204 0
2025-12-05 20:02:13 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: e13cbd1a-eb43-5b61-bf64-2ddc1cfdeb85 with identifier: 1788842
2025-12-05 20:02:13 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:02:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:02:13 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: e13cbd1a-eb43-5b61-bf64-2ddc1cfdeb85 with identifier: 1788842
2025-12-05 20:02:13 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: e13cbd1a-eb43-5b61-bf64-2ddc1cfdeb85 with identifier: 1788842 to the API
2025-12-05 20:02:13 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:14 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:14 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/763/6aa1ea1704934a78b4c2bd59e8ce00c8/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 137,
 'color': 'Cristal',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'e13cbd1a-eb43-5b61-bf64-2ddc1cfdeb85',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'DS AUTOMOBILES',
 'model': 'DS 7 CROSSBACK',
 'odometer': 123667,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_e13cbd1a-eb43-5b61-bf64-2ddc1cfdeb85/e13cbd1a-eb43-5b61-bf64-2ddc1cfdeb85_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 377795,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/763/6aa1ea1704934a78b4c2bd59e8ce00c8/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-10-04',
 'seats_number': 5,
 'seller_name': None,
 'title': 'DS AUTOMOBILES DS 7 CROSSBACK BlueHDi DE 96kW (130CV) AT. PERF.LINE',
 'transmission': 'manual',
 'trim': 'BlueHDi DE 96kW (130CV) AT. PERF.LINE',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788842%27 HTTP/1.1" 200 None
2025-12-05 20:02:14 [ayvens.es] INFO: Saving data for 1788842: {'created_time': 1764964934.210255, 'last_price_update_time': 1764964934.210266, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788842') HTTP/1.1" 204 0
2025-12-05 20:02:14 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 3a1f98d1-ae9c-50f1-892f-4dede0081e73 with identifier: 1788841
2025-12-05 20:02:14 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 3a1f98d1-ae9c-50f1-892f-4dede0081e73 with identifier: 1788841
2025-12-05 20:02:14 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: 155826 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    155826
2025-12-05 20:02:14 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE_FOR_COUNTRY failed for item: 3a1f98d1-ae9c-50f1-892f-4dede0081e73 spider: ayvens.es
2025-12-05 20:02:14 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 3a1f98d1-ae9c-50f1-892f-4dede0081e73 due to rule: high_mileage_for_country.json, error: 155826 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    155826
{'auction_title': '',
 'c_o_2_emission_value': 125,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '3a1f98d1-ae9c-50f1-892f-4dede0081e73',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'PASSAT',
 'odometer': 155826,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_3a1f98d1-ae9c-50f1-892f-4dede0081e73/3a1f98d1-ae9c-50f1-892f-4dede0081e73_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 609174,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/030/64221da2216d41658d3d68ceee9a2cbc/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-03-12',
 'seats_number': 5,
 'seller_name': None,
 'title': 'VOLKSWAGEN PASSAT Executive 2.0 TDI 90kW (122CV) DSG',
 'transmission': 'manual',
 'trim': 'Executive 2.0 TDI 90kW (122CV) DSG',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788841%27 HTTP/1.1" 200 None
2025-12-05 20:02:14 [ayvens.es] INFO: Saving data for 1788841: {'created_time': 1764964934.466855, 'last_price_update_time': 1764964934.466864, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788841') HTTP/1.1" 204 0
2025-12-05 20:02:14 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 31b01c57-dcc1-5747-bd00-30b51bf5ddf9 with identifier: 1788844
2025-12-05 20:02:14 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 31b01c57-dcc1-5747-bd00-30b51bf5ddf9 with identifier: 1788844
2025-12-05 20:02:14 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 31b01c57-dcc1-5747-bd00-30b51bf5ddf9 with identifier: 1788844 to the API
2025-12-05 20:02:14 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:14 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:14 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/086/a6af760630274cc39e470c74fffd0f5c/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 120,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '31b01c57-dcc1-5747-bd00-30b51bf5ddf9',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'CITROËN',
 'model': 'C3 AIRCROSS',
 'odometer': 60044,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_31b01c57-dcc1-5747-bd00-30b51bf5ddf9/31b01c57-dcc1-5747-bd00-30b51bf5ddf9_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 328274,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/086/a6af760630274cc39e470c74fffd0f5c/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2022-10-28',
 'seats_number': 5,
 'seller_name': None,
 'title': 'CITROËN C3 AIRCROSS BlueHDi 81kW (110CV) S&S Feel Pack',
 'transmission': 'manual',
 'trim': 'BlueHDi 81kW (110CV) S&S Feel Pack',
 'vin': None,
 'year': 2022}
2025-12-05 20:02:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788844%27 HTTP/1.1" 200 None
2025-12-05 20:02:14 [ayvens.es] INFO: Saving data for 1788844: {'created_time': 1764964934.909803, 'last_price_update_time': 1764964934.909815, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788844') HTTP/1.1" 204 0
2025-12-05 20:02:14 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 970c6591-58a7-546c-bf2d-a3f9971ad9c4 with identifier: 1788845
2025-12-05 20:02:14 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 970c6591-58a7-546c-bf2d-a3f9971ad9c4 with identifier: 1788845
2025-12-05 20:02:15 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'renault' containing 'renault' and model 'kangoo express' containing 'express'
2025-12-05 20:02:15 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 970c6591-58a7-546c-bf2d-a3f9971ad9c4 spider: ayvens.es
2025-12-05 20:02:15 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 970c6591-58a7-546c-bf2d-a3f9971ad9c4 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'renault' containing 'renault' and model 'kangoo express' containing 'express'
{'auction_title': '',
 'c_o_2_emission_value': 123,
 'color': 'Amarillo',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '970c6591-58a7-546c-bf2d-a3f9971ad9c4',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'KANGOO EXPRESS',
 'odometer': 78696,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_970c6591-58a7-546c-bf2d-a3f9971ad9c4/970c6591-58a7-546c-bf2d-a3f9971ad9c4_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 357941,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/635/0c875a59d75e4fe1bce46645419dbc3c/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-08-01',
 'seats_number': 2,
 'seller_name': None,
 'title': 'RENAULT KANGOO EXPRESS (O) Profesional Maxi 2p dCi 66 kW (90 CV '
          '**REVISAR PERITACION**',
 'transmission': 'manual',
 'trim': '(O) Profesional Maxi 2p dCi 66 kW (90 CV **REVISAR PERITACION**',
 'vin': None,
 'year': 2019}
2025-12-05 20:02:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788845%27 HTTP/1.1" 200 None
2025-12-05 20:02:15 [ayvens.es] INFO: Saving data for 1788845: {'created_time': 1764964935.180425, 'last_price_update_time': 1764964935.180436, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788845') HTTP/1.1" 204 0
2025-12-05 20:02:15 [ayvens.es] INFO: Found listing with ID: 1788854
2025-12-05 20:02:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788854%27 HTTP/1.1" 200 None
2025-12-05 20:02:15 [ayvens.es] INFO: Scrape type for 1788854: 1
2025-12-05 20:02:15 [ayvens.es] INFO: Extracting data for car ID: 1788854
2025-12-05 20:02:15 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 43914037-7635-533e-bb58-b6c7e26a451a with identifier: 1788854
2025-12-05 20:02:15 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 43914037-7635-533e-bb58-b6c7e26a451a with scrape type 1
2025-12-05 20:02:15 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 43914037-7635-533e-bb58-b6c7e26a451a
2025-12-05 20:02:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F482%252F3ad3209cbaf94b6d8302d09bdd21b930%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:15 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/482/3ad3209cbaf94b6d8302d09bdd21b930/images/large/1.jpg not downloaded yet
2025-12-05 20:02:15 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/582/1926aa357d714e15ba1791530bd7c090/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:15 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:15 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:15 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:15 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0568890>
2025-12-05 20:02:15 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:15 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:15 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:02:15 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:15 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:15 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:15 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_ad487e32-9ed9-51c0-b6f3-120ad68435c9/ad487e32-9ed9-51c0-b6f3-120ad68435c9_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F796%252Fe87f095dfbd9408094b966298820d438%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:15 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/796/e87f095dfbd9408094b966298820d438/images/large/1.jpg saved to cache
2025-12-05 20:02:15 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item ad487e32-9ed9-51c0-b6f3-120ad68435c9
2025-12-05 20:02:15 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: ad487e32-9ed9-51c0-b6f3-120ad68435c9, skipping ID generation
2025-12-05 20:02:15 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ad487e32-9ed9-51c0-b6f3-120ad68435c9 with scrape type 1
2025-12-05 20:02:15 [crawlers.middlewares.photo_download_middleware] INFO: Removing item ad487e32-9ed9-51c0-b6f3-120ad68435c9 from tracking dictionaries
2025-12-05 20:02:15 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item ad487e32-9ed9-51c0-b6f3-120ad68435c9 sending to next pipeline
2025-12-05 20:02:15 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: ad487e32-9ed9-51c0-b6f3-120ad68435c9, identifier: 1788846
2025-12-05 20:02:15 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:15 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:15 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:15 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0699b50>
2025-12-05 20:02:15 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:15 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:15 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:02:15 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:15 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:15 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:15 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_120a4e5e-f749-5270-a76d-0ddc77285650/120a4e5e-f749-5270-a76d-0ddc77285650_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F110%252Fd85cda5b851245a58e04a8a144625151%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:16 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/110/d85cda5b851245a58e04a8a144625151/images/large/1.jpg saved to cache
2025-12-05 20:02:16 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 120a4e5e-f749-5270-a76d-0ddc77285650
2025-12-05 20:02:16 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 120a4e5e-f749-5270-a76d-0ddc77285650, skipping ID generation
2025-12-05 20:02:16 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 120a4e5e-f749-5270-a76d-0ddc77285650 with scrape type 1
2025-12-05 20:02:16 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 120a4e5e-f749-5270-a76d-0ddc77285650 from tracking dictionaries
2025-12-05 20:02:16 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 120a4e5e-f749-5270-a76d-0ddc77285650 sending to next pipeline
2025-12-05 20:02:16 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 120a4e5e-f749-5270-a76d-0ddc77285650, identifier: 1788847
2025-12-05 20:02:16 [ayvens.es] INFO: Found listing with ID: 1788855
2025-12-05 20:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788855%27 HTTP/1.1" 200 None
2025-12-05 20:02:16 [ayvens.es] INFO: Scrape type for 1788855: 1
2025-12-05 20:02:16 [ayvens.es] INFO: Extracting data for car ID: 1788855
2025-12-05 20:02:16 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 130d34c1-ed5d-5c6b-b1f4-ea17b5728936 with identifier: 1788855
2025-12-05 20:02:16 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 130d34c1-ed5d-5c6b-b1f4-ea17b5728936 with scrape type 1
2025-12-05 20:02:16 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 130d34c1-ed5d-5c6b-b1f4-ea17b5728936
2025-12-05 20:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F383%252F0b821254a0f749fab0092d24858ebc5a%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:16 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/383/0b821254a0f749fab0092d24858ebc5a/images/large/1.jpg not downloaded yet
2025-12-05 20:02:16 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: ad487e32-9ed9-51c0-b6f3-120ad68435c9 with identifier: 1788846
2025-12-05 20:02:16 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: ad487e32-9ed9-51c0-b6f3-120ad68435c9 with identifier: 1788846
2025-12-05 20:02:16 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'toyota' containing 'toyota' and model 'hilux' containing 'hilux'
2025-12-05 20:02:16 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: ad487e32-9ed9-51c0-b6f3-120ad68435c9 spider: ayvens.es
2025-12-05 20:02:16 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: ad487e32-9ed9-51c0-b6f3-120ad68435c9 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'toyota' containing 'toyota' and model 'hilux' containing 'hilux'
{'auction_title': '',
 'c_o_2_emission_value': 185,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '2.4L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'ad487e32-9ed9-51c0-b6f3-120ad68435c9',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'TOYOTA',
 'model': 'HILUX',
 'odometer': 53245,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ad487e32-9ed9-51c0-b6f3-120ad68435c9/ad487e32-9ed9-51c0-b6f3-120ad68435c9_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 340018,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/796/e87f095dfbd9408094b966298820d438/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-01-02',
 'seats_number': 5,
 'seller_name': None,
 'title': 'TOYOTA HILUX 2.4 D-4D Cabina Doble GX **REVISAR PERITACION**',
 'transmission': 'manual',
 'trim': '2.4 D-4D Cabina Doble GX **REVISAR PERITACION**',
 'vin': None,
 'year': 2019}
2025-12-05 20:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788846%27 HTTP/1.1" 200 None
2025-12-05 20:02:16 [ayvens.es] INFO: Saving data for 1788846: {'created_time': 1764964936.369742, 'last_price_update_time': 1764964936.369751, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788846') HTTP/1.1" 204 0
2025-12-05 20:02:16 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 120a4e5e-f749-5270-a76d-0ddc77285650 with identifier: 1788847
2025-12-05 20:02:16 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 120a4e5e-f749-5270-a76d-0ddc77285650 with identifier: 1788847
2025-12-05 20:02:16 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 120a4e5e-f749-5270-a76d-0ddc77285650 with identifier: 1788847 to the API
2025-12-05 20:02:16 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:16 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:16 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/110/d85cda5b851245a58e04a8a144625151/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 132,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '120a4e5e-f749-5270-a76d-0ddc77285650',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'AUDI',
 'model': 'Q3',
 'odometer': 33661,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_120a4e5e-f749-5270-a76d-0ddc77285650/120a4e5e-f749-5270-a76d-0ddc77285650_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 353513,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/110/d85cda5b851245a58e04a8a144625151/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-12-07',
 'seats_number': 5,
 'seller_name': None,
 'title': 'AUDI Q3 Advanced 35 TDI 110kW (150CV)',
 'transmission': 'manual',
 'trim': 'Advanced 35 TDI 110kW (150CV)',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788847%27 HTTP/1.1" 200 None
2025-12-05 20:02:16 [ayvens.es] INFO: Saving data for 1788847: {'created_time': 1764964936.706638, 'last_price_update_time': 1764964936.706652, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788847') HTTP/1.1" 204 0
2025-12-05 20:02:16 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/868/533e3ae516f04cbfa41f3a4374173671/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:16 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/524/8453b2cb5d6c43e588c3528f6ee67ad9/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:16 [ayvens.es] INFO: Found listing with ID: 1788856
2025-12-05 20:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788856%27 HTTP/1.1" 200 None
2025-12-05 20:02:16 [ayvens.es] INFO: Scrape type for 1788856: 1
2025-12-05 20:02:16 [ayvens.es] INFO: Extracting data for car ID: 1788856
2025-12-05 20:02:16 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: d0407b1f-9376-5dd6-9cb1-3be59138a6bc with identifier: 1788856
2025-12-05 20:02:16 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item d0407b1f-9376-5dd6-9cb1-3be59138a6bc with scrape type 1
2025-12-05 20:02:16 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item d0407b1f-9376-5dd6-9cb1-3be59138a6bc
2025-12-05 20:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F168%252F6bec3f99eec64852857e7af6e2114b18%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:16 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/168/6bec3f99eec64852857e7af6e2114b18/images/large/1.jpg not downloaded yet
2025-12-05 20:02:16 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:16 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:16 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:16 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b0e490>
2025-12-05 20:02:16 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:16 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:16 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:02:16 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:17 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_c3b6bbbb-79bb-5038-ae4e-40d22d319033/c3b6bbbb-79bb-5038-ae4e-40d22d319033_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F582%252F1926aa357d714e15ba1791530bd7c090%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:17 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/582/1926aa357d714e15ba1791530bd7c090/images/large/1.jpg saved to cache
2025-12-05 20:02:17 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item c3b6bbbb-79bb-5038-ae4e-40d22d319033
2025-12-05 20:02:17 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: c3b6bbbb-79bb-5038-ae4e-40d22d319033, skipping ID generation
2025-12-05 20:02:17 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item c3b6bbbb-79bb-5038-ae4e-40d22d319033 with scrape type 1
2025-12-05 20:02:17 [crawlers.middlewares.photo_download_middleware] INFO: Removing item c3b6bbbb-79bb-5038-ae4e-40d22d319033 from tracking dictionaries
2025-12-05 20:02:17 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item c3b6bbbb-79bb-5038-ae4e-40d22d319033 sending to next pipeline
2025-12-05 20:02:17 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: c3b6bbbb-79bb-5038-ae4e-40d22d319033, identifier: 1788848
2025-12-05 20:02:17 [ayvens.es] INFO: Found listing with ID: 1788857
2025-12-05 20:02:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788857%27 HTTP/1.1" 200 None
2025-12-05 20:02:17 [ayvens.es] INFO: Scrape type for 1788857: 1
2025-12-05 20:02:17 [ayvens.es] INFO: Extracting data for car ID: 1788857
2025-12-05 20:02:17 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 647645f7-9623-52d5-8508-a7fec369d75d with identifier: 1788857
2025-12-05 20:02:17 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 647645f7-9623-52d5-8508-a7fec369d75d with scrape type 1
2025-12-05 20:02:17 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 647645f7-9623-52d5-8508-a7fec369d75d
2025-12-05 20:02:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F533%252F70ee3ebeb2bb4706a6a34a3a34b709a2%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:17 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/533/70ee3ebeb2bb4706a6a34a3a34b709a2/images/large/1.jpg not downloaded yet
2025-12-05 20:02:17 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: c3b6bbbb-79bb-5038-ae4e-40d22d319033 with identifier: 1788848
2025-12-05 20:02:17 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: c3b6bbbb-79bb-5038-ae4e-40d22d319033 with identifier: 1788848
2025-12-05 20:02:17 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'renault' containing 'renault' and model 'kangoo express' containing 'express'
2025-12-05 20:02:17 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: c3b6bbbb-79bb-5038-ae4e-40d22d319033 spider: ayvens.es
2025-12-05 20:02:17 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: c3b6bbbb-79bb-5038-ae4e-40d22d319033 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'renault' containing 'renault' and model 'kangoo express' containing 'express'
{'auction_title': '',
 'c_o_2_emission_value': 117,
 'color': 'Pintura',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'c3b6bbbb-79bb-5038-ae4e-40d22d319033',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'KANGOO EXPRESS',
 'odometer': 90595,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_c3b6bbbb-79bb-5038-ae4e-40d22d319033/c3b6bbbb-79bb-5038-ae4e-40d22d319033_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 448710,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/582/1926aa357d714e15ba1791530bd7c090/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-07-24',
 'seats_number': 2,
 'seller_name': None,
 'title': 'RENAULT KANGOO EXPRESS Maxi 2p Blue dCi 70 kW (95CV)',
 'transmission': 'manual',
 'trim': 'Maxi 2p Blue dCi 70 kW (95CV)',
 'vin': None,
 'year': 2020}
2025-12-05 20:02:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788848%27 HTTP/1.1" 200 None
2025-12-05 20:02:17 [ayvens.es] INFO: Saving data for 1788848: {'created_time': 1764964937.555445, 'last_price_update_time': 1764964937.555456, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788848') HTTP/1.1" 204 0
2025-12-05 20:02:17 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/981/738216625f66483b9377f1c23b8a7790/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:17 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/249/18a8de782fbf4710968a365afc256761/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:17 [ayvens.es] INFO: Found listing with ID: 1788858
2025-12-05 20:02:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788858%27 HTTP/1.1" 200 None
2025-12-05 20:02:17 [ayvens.es] INFO: Scrape type for 1788858: 1
2025-12-05 20:02:17 [ayvens.es] INFO: Extracting data for car ID: 1788858
2025-12-05 20:02:17 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 43e1ac88-e2f6-57c4-8e1f-3a361adf1655 with identifier: 1788858
2025-12-05 20:02:17 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 43e1ac88-e2f6-57c4-8e1f-3a361adf1655 with scrape type 1
2025-12-05 20:02:17 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 43e1ac88-e2f6-57c4-8e1f-3a361adf1655
2025-12-05 20:02:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F926%252F98ef3930cfc54214ad0f470c17dd29a5%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:17 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/926/98ef3930cfc54214ad0f470c17dd29a5/images/large/1.jpg not downloaded yet
2025-12-05 20:02:17 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:17 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:17 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:17 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0578210>
2025-12-05 20:02:17 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:17 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:17 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:02:17 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:17 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_c4e9a70a-cdd6-55dd-a511-bca4aaee7393/c4e9a70a-cdd6-55dd-a511-bca4aaee7393_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F868%252F533e3ae516f04cbfa41f3a4374173671%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:17 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/868/533e3ae516f04cbfa41f3a4374173671/images/large/1.jpg saved to cache
2025-12-05 20:02:17 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item c4e9a70a-cdd6-55dd-a511-bca4aaee7393
2025-12-05 20:02:17 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: c4e9a70a-cdd6-55dd-a511-bca4aaee7393, skipping ID generation
2025-12-05 20:02:17 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item c4e9a70a-cdd6-55dd-a511-bca4aaee7393 with scrape type 1
2025-12-05 20:02:17 [crawlers.middlewares.photo_download_middleware] INFO: Removing item c4e9a70a-cdd6-55dd-a511-bca4aaee7393 from tracking dictionaries
2025-12-05 20:02:17 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item c4e9a70a-cdd6-55dd-a511-bca4aaee7393 sending to next pipeline
2025-12-05 20:02:18 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: c4e9a70a-cdd6-55dd-a511-bca4aaee7393, identifier: 1788849
2025-12-05 20:02:18 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:18 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:18 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:18 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0568a10>
2025-12-05 20:02:18 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:18 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:18 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:02:18 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:18 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:18 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:18 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_bdecba64-e44e-5683-8a9b-cbddfaf80ff7/bdecba64-e44e-5683-8a9b-cbddfaf80ff7_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F524%252F8453b2cb5d6c43e588c3528f6ee67ad9%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/524/8453b2cb5d6c43e588c3528f6ee67ad9/images/large/1.jpg saved to cache
2025-12-05 20:02:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item bdecba64-e44e-5683-8a9b-cbddfaf80ff7
2025-12-05 20:02:18 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: bdecba64-e44e-5683-8a9b-cbddfaf80ff7, skipping ID generation
2025-12-05 20:02:18 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item bdecba64-e44e-5683-8a9b-cbddfaf80ff7 with scrape type 1
2025-12-05 20:02:18 [crawlers.middlewares.photo_download_middleware] INFO: Removing item bdecba64-e44e-5683-8a9b-cbddfaf80ff7 from tracking dictionaries
2025-12-05 20:02:18 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item bdecba64-e44e-5683-8a9b-cbddfaf80ff7 sending to next pipeline
2025-12-05 20:02:18 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: bdecba64-e44e-5683-8a9b-cbddfaf80ff7, identifier: 1788850
2025-12-05 20:02:18 [ayvens.es] INFO: Found listing with ID: 1788859
2025-12-05 20:02:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788859%27 HTTP/1.1" 200 None
2025-12-05 20:02:18 [ayvens.es] INFO: Scrape type for 1788859: 1
2025-12-05 20:02:18 [ayvens.es] INFO: Extracting data for car ID: 1788859
2025-12-05 20:02:18 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 5d32e842-4e5b-5de8-840d-5b438ec73f75 with identifier: 1788859
2025-12-05 20:02:18 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 5d32e842-4e5b-5de8-840d-5b438ec73f75 with scrape type 1
2025-12-05 20:02:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 5d32e842-4e5b-5de8-840d-5b438ec73f75
2025-12-05 20:02:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F398%252F955c51bad48e4157bd5035b12d973999%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/398/955c51bad48e4157bd5035b12d973999/images/large/1.jpg not downloaded yet
2025-12-05 20:02:18 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: c4e9a70a-cdd6-55dd-a511-bca4aaee7393 with identifier: 1788849
2025-12-05 20:02:18 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: c4e9a70a-cdd6-55dd-a511-bca4aaee7393 with identifier: 1788849
2025-12-05 20:02:18 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: c4e9a70a-cdd6-55dd-a511-bca4aaee7393 with identifier: 1788849 to the API
2025-12-05 20:02:18 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:18 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:18 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/868/533e3ae516f04cbfa41f3a4374173671/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 113,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'c4e9a70a-cdd6-55dd-a511-bca4aaee7393',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SEAT',
 'model': 'LEON',
 'odometer': 104525,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_c4e9a70a-cdd6-55dd-a511-bca4aaee7393/c4e9a70a-cdd6-55dd-a511-bca4aaee7393_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 327791,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/868/533e3ae516f04cbfa41f3a4374173671/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-10-15',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SEAT LEON 2.0 TDI 85kW S&S Style Go',
 'transmission': 'manual',
 'trim': '2.0 TDI 85kW S&S Style Go',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788849%27 HTTP/1.1" 200 None
2025-12-05 20:02:18 [ayvens.es] INFO: Saving data for 1788849: {'created_time': 1764964938.889456, 'last_price_update_time': 1764964938.889465, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788849') HTTP/1.1" 204 0
2025-12-05 20:02:19 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: bdecba64-e44e-5683-8a9b-cbddfaf80ff7 with identifier: 1788850
2025-12-05 20:02:19 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: bdecba64-e44e-5683-8a9b-cbddfaf80ff7 with identifier: 1788850
2025-12-05 20:02:19 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: bdecba64-e44e-5683-8a9b-cbddfaf80ff7 with identifier: 1788850 to the API
2025-12-05 20:02:19 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:19 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:19 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/524/8453b2cb5d6c43e588c3528f6ee67ad9/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 113,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'bdecba64-e44e-5683-8a9b-cbddfaf80ff7',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SEAT',
 'model': 'LEON',
 'odometer': 98786,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_bdecba64-e44e-5683-8a9b-cbddfaf80ff7/bdecba64-e44e-5683-8a9b-cbddfaf80ff7_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 329192,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/524/8453b2cb5d6c43e588c3528f6ee67ad9/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-07-06',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SEAT LEON 2.0 TDI 85kW S&S Style Go',
 'transmission': 'manual',
 'trim': '2.0 TDI 85kW S&S Style Go',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788850%27 HTTP/1.1" 200 None
2025-12-05 20:02:19 [ayvens.es] INFO: Saving data for 1788850: {'created_time': 1764964939.3769, 'last_price_update_time': 1764964939.376911, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788850') HTTP/1.1" 204 0
2025-12-05 20:02:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/282/93a166eb803a4694aa407909176a4f38/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:19 [ayvens.es] INFO: Found listing with ID: 1788860
2025-12-05 20:02:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788860%27 HTTP/1.1" 200 None
2025-12-05 20:02:19 [ayvens.es] INFO: Scrape type for 1788860: 1
2025-12-05 20:02:19 [ayvens.es] INFO: Extracting data for car ID: 1788860
2025-12-05 20:02:19 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: b795cc83-6a67-5cf7-a253-d0215447f3ef with identifier: 1788860
2025-12-05 20:02:19 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b795cc83-6a67-5cf7-a253-d0215447f3ef with scrape type 1
2025-12-05 20:02:19 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item b795cc83-6a67-5cf7-a253-d0215447f3ef
2025-12-05 20:02:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F950%252F55c8537402424169b7df219b2f7e5ab2%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/950/55c8537402424169b7df219b2f7e5ab2/images/large/1.jpg not downloaded yet
2025-12-05 20:02:19 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:19 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:19 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:19 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a81f50>
2025-12-05 20:02:19 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:19 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:19 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1969
2025-12-05 20:02:19 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:19 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:19 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:19 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_2f7443d0-fa37-57c5-a327-4a7cf275dee2/2f7443d0-fa37-57c5-a327-4a7cf275dee2_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F981%252F738216625f66483b9377f1c23b8a7790%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/981/738216625f66483b9377f1c23b8a7790/images/large/1.jpg saved to cache
2025-12-05 20:02:19 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 2f7443d0-fa37-57c5-a327-4a7cf275dee2
2025-12-05 20:02:19 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 2f7443d0-fa37-57c5-a327-4a7cf275dee2, skipping ID generation
2025-12-05 20:02:19 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 2f7443d0-fa37-57c5-a327-4a7cf275dee2 with scrape type 1
2025-12-05 20:02:19 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 2f7443d0-fa37-57c5-a327-4a7cf275dee2 from tracking dictionaries
2025-12-05 20:02:19 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 2f7443d0-fa37-57c5-a327-4a7cf275dee2 sending to next pipeline
2025-12-05 20:02:19 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 2f7443d0-fa37-57c5-a327-4a7cf275dee2, identifier: 1788852
2025-12-05 20:02:19 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:19 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:19 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:19 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0c3f090>
2025-12-05 20:02:19 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:19 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:20 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1971
2025-12-05 20:02:20 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:20 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:20 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:20 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_919f3336-9283-5ec1-b889-5db63c0c2967/919f3336-9283-5ec1-b889-5db63c0c2967_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F249%252F18a8de782fbf4710968a365afc256761%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/249/18a8de782fbf4710968a365afc256761/images/large/1.jpg saved to cache
2025-12-05 20:02:20 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 919f3336-9283-5ec1-b889-5db63c0c2967
2025-12-05 20:02:20 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 919f3336-9283-5ec1-b889-5db63c0c2967, skipping ID generation
2025-12-05 20:02:20 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 919f3336-9283-5ec1-b889-5db63c0c2967 with scrape type 1
2025-12-05 20:02:20 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 919f3336-9283-5ec1-b889-5db63c0c2967 from tracking dictionaries
2025-12-05 20:02:20 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 919f3336-9283-5ec1-b889-5db63c0c2967 sending to next pipeline
2025-12-05 20:02:20 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 919f3336-9283-5ec1-b889-5db63c0c2967, identifier: 1788851
2025-12-05 20:02:20 [ayvens.es] INFO: Found listing with ID: 1788861
2025-12-05 20:02:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788861%27 HTTP/1.1" 200 None
2025-12-05 20:02:20 [ayvens.es] INFO: Scrape type for 1788861: 1
2025-12-05 20:02:20 [ayvens.es] INFO: Extracting data for car ID: 1788861
2025-12-05 20:02:20 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 38d68140-8304-5b0f-9632-11d274139ca3 with identifier: 1788861
2025-12-05 20:02:20 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 38d68140-8304-5b0f-9632-11d274139ca3 with scrape type 1
2025-12-05 20:02:20 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 38d68140-8304-5b0f-9632-11d274139ca3
2025-12-05 20:02:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F887%252Fa1e83022113e4852ad1d71a63b613221%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/887/a1e83022113e4852ad1d71a63b613221/images/large/1.jpg not downloaded yet
2025-12-05 20:02:20 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 2f7443d0-fa37-57c5-a327-4a7cf275dee2 with identifier: 1788852
2025-12-05 20:02:20 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 2f7443d0-fa37-57c5-a327-4a7cf275dee2 with identifier: 1788852
2025-12-05 20:02:20 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: 199119 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    199119
2025-12-05 20:02:20 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE_FOR_COUNTRY failed for item: 2f7443d0-fa37-57c5-a327-4a7cf275dee2 spider: ayvens.es
2025-12-05 20:02:20 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 2f7443d0-fa37-57c5-a327-4a7cf275dee2 due to rule: high_mileage_for_country.json, error: 199119 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    199119
{'auction_title': '',
 'c_o_2_emission_value': 128,
 'color': 'Saphirschwarz',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '2f7443d0-fa37-57c5-a327-4a7cf275dee2',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'BMW',
 'model': 'SERIES 4',
 'odometer': 199119,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_2f7443d0-fa37-57c5-a327-4a7cf275dee2/2f7443d0-fa37-57c5-a327-4a7cf275dee2_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 350317,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/981/738216625f66483b9377f1c23b8a7790/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-05-28',
 'seats_number': 5,
 'seller_name': None,
 'title': 'BMW SERIES 4 (E) 420d Gran Coupe',
 'transmission': 'manual',
 'trim': '(E) 420d Gran Coupe',
 'vin': None,
 'year': 2020}
2025-12-05 20:02:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788852%27 HTTP/1.1" 200 None
2025-12-05 20:02:20 [ayvens.es] INFO: Saving data for 1788852: {'created_time': 1764964940.630383, 'last_price_update_time': 1764964940.630393, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788852') HTTP/1.1" 204 0
2025-12-05 20:02:20 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 919f3336-9283-5ec1-b889-5db63c0c2967 with identifier: 1788851
2025-12-05 20:02:20 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 919f3336-9283-5ec1-b889-5db63c0c2967 with identifier: 1788851
2025-12-05 20:02:20 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 919f3336-9283-5ec1-b889-5db63c0c2967 with identifier: 1788851 to the API
2025-12-05 20:02:20 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:20 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:21 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/249/18a8de782fbf4710968a365afc256761/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 127,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '919f3336-9283-5ec1-b889-5db63c0c2967',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'AUDI',
 'model': 'A4',
 'odometer': 88924,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_919f3336-9283-5ec1-b889-5db63c0c2967/919f3336-9283-5ec1-b889-5db63c0c2967_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 376319,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/249/18a8de782fbf4710968a365afc256761/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-10-26',
 'seats_number': 5,
 'seller_name': None,
 'title': 'AUDI A4 Avant Advanced 35 TDI 120kW S tronic',
 'transmission': 'manual',
 'trim': 'Avant Advanced 35 TDI 120kW S tronic',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788851%27 HTTP/1.1" 200 None
2025-12-05 20:02:21 [ayvens.es] INFO: Saving data for 1788851: {'created_time': 1764964941.019781, 'last_price_update_time': 1764964941.019791, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788851') HTTP/1.1" 204 0
2025-12-05 20:02:21 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/482/3ad3209cbaf94b6d8302d09bdd21b930/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:21 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/168/6bec3f99eec64852857e7af6e2114b18/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:21 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/383/0b821254a0f749fab0092d24858ebc5a/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:21 [ayvens.es] INFO: Found listing with ID: 1788862
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788862%27 HTTP/1.1" 200 None
2025-12-05 20:02:21 [ayvens.es] INFO: Scrape type for 1788862: 1
2025-12-05 20:02:21 [ayvens.es] INFO: Extracting data for car ID: 1788862
2025-12-05 20:02:21 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 789dfe9a-fc34-5022-bb66-ec7da309f4c9 with identifier: 1788862
2025-12-05 20:02:21 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 789dfe9a-fc34-5022-bb66-ec7da309f4c9 with scrape type 1
2025-12-05 20:02:21 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 789dfe9a-fc34-5022-bb66-ec7da309f4c9
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F360%252F5e72d1785c2741a88879d1cbc48c7006%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/360/5e72d1785c2741a88879d1cbc48c7006/images/large/1.jpg not downloaded yet
2025-12-05 20:02:21 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:21 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:21 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:21 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b2b22890>
2025-12-05 20:02:21 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:21 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_d694b55c-c504-5406-93ef-7b8370add0fd/d694b55c-c504-5406-93ef-7b8370add0fd_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F282%252F93a166eb803a4694aa407909176a4f38%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/282/93a166eb803a4694aa407909176a4f38/images/large/1.jpg saved to cache
2025-12-05 20:02:21 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item d694b55c-c504-5406-93ef-7b8370add0fd
2025-12-05 20:02:21 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: d694b55c-c504-5406-93ef-7b8370add0fd, skipping ID generation
2025-12-05 20:02:21 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item d694b55c-c504-5406-93ef-7b8370add0fd with scrape type 1
2025-12-05 20:02:21 [crawlers.middlewares.photo_download_middleware] INFO: Removing item d694b55c-c504-5406-93ef-7b8370add0fd from tracking dictionaries
2025-12-05 20:02:21 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item d694b55c-c504-5406-93ef-7b8370add0fd sending to next pipeline
2025-12-05 20:02:21 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: d694b55c-c504-5406-93ef-7b8370add0fd, identifier: 1788853
2025-12-05 20:02:21 [ayvens.es] INFO: Found listing with ID: 1788863
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788863%27 HTTP/1.1" 200 None
2025-12-05 20:02:21 [ayvens.es] INFO: Scrape type for 1788863: 1
2025-12-05 20:02:21 [ayvens.es] INFO: Extracting data for car ID: 1788863
2025-12-05 20:02:21 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 8ebf955d-a20c-5671-b328-40af0c866d5b with identifier: 1788863
2025-12-05 20:02:21 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 8ebf955d-a20c-5671-b328-40af0c866d5b with scrape type 1
2025-12-05 20:02:21 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 8ebf955d-a20c-5671-b328-40af0c866d5b
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F279%252F55c3f0f83ba64b29a525c7d6872d355e%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/279/55c3f0f83ba64b29a525c7d6872d355e/images/large/1.jpg not downloaded yet
2025-12-05 20:02:21 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: d694b55c-c504-5406-93ef-7b8370add0fd with identifier: 1788853
2025-12-05 20:02:21 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: d694b55c-c504-5406-93ef-7b8370add0fd with identifier: 1788853
2025-12-05 20:02:21 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: d694b55c-c504-5406-93ef-7b8370add0fd with identifier: 1788853 to the API
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:21 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/282/93a166eb803a4694aa407909176a4f38/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 130,
 'color': 'Saphirschwarz',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'd694b55c-c504-5406-93ef-7b8370add0fd',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'BMW',
 'model': 'X1',
 'odometer': 29543,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_d694b55c-c504-5406-93ef-7b8370add0fd/d694b55c-c504-5406-93ef-7b8370add0fd_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 366833,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/282/93a166eb803a4694aa407909176a4f38/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-07-20',
 'seats_number': 5,
 'seller_name': None,
 'title': 'BMW X1 sDrive18d',
 'transmission': 'manual',
 'trim': 'sDrive18d',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788853%27 HTTP/1.1" 200 None
2025-12-05 20:02:21 [ayvens.es] INFO: Saving data for 1788853: {'created_time': 1764964941.913163, 'last_price_update_time': 1764964941.913173, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788853') HTTP/1.1" 204 0
2025-12-05 20:02:21 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/533/70ee3ebeb2bb4706a6a34a3a34b709a2/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:21 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/926/98ef3930cfc54214ad0f470c17dd29a5/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:21 [ayvens.es] INFO: Found listing with ID: 1788864
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788864%27 HTTP/1.1" 200 None
2025-12-05 20:02:21 [ayvens.es] INFO: Scrape type for 1788864: 1
2025-12-05 20:02:21 [ayvens.es] INFO: Extracting data for car ID: 1788864
2025-12-05 20:02:21 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 491b615f-2d91-561d-99e4-fb1c9ad45cc7 with identifier: 1788864
2025-12-05 20:02:21 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 491b615f-2d91-561d-99e4-fb1c9ad45cc7 with scrape type 1
2025-12-05 20:02:21 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 491b615f-2d91-561d-99e4-fb1c9ad45cc7
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F240%252F5e5bf5b5f2284a1faf5a46071c78b58c%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/240/5e5bf5b5f2284a1faf5a46071c78b58c/images/large/1.jpg not downloaded yet
2025-12-05 20:02:21 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:21 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:21 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:21 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0adad50>
2025-12-05 20:02:21 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:21 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:22 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:02:22 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:22 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:22 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:22 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_43914037-7635-533e-bb58-b6c7e26a451a/43914037-7635-533e-bb58-b6c7e26a451a_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F482%252F3ad3209cbaf94b6d8302d09bdd21b930%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/482/3ad3209cbaf94b6d8302d09bdd21b930/images/large/1.jpg saved to cache
2025-12-05 20:02:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 43914037-7635-533e-bb58-b6c7e26a451a
2025-12-05 20:02:22 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 43914037-7635-533e-bb58-b6c7e26a451a, skipping ID generation
2025-12-05 20:02:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 43914037-7635-533e-bb58-b6c7e26a451a with scrape type 1
2025-12-05 20:02:22 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 43914037-7635-533e-bb58-b6c7e26a451a from tracking dictionaries
2025-12-05 20:02:22 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 43914037-7635-533e-bb58-b6c7e26a451a sending to next pipeline
2025-12-05 20:02:22 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 43914037-7635-533e-bb58-b6c7e26a451a, identifier: 1788854
2025-12-05 20:02:22 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:22 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:22 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:22 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0df09d0>
2025-12-05 20:02:22 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:22 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:22 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:02:22 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:22 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:22 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:22 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_d0407b1f-9376-5dd6-9cb1-3be59138a6bc/d0407b1f-9376-5dd6-9cb1-3be59138a6bc_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F168%252F6bec3f99eec64852857e7af6e2114b18%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/168/6bec3f99eec64852857e7af6e2114b18/images/large/1.jpg saved to cache
2025-12-05 20:02:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item d0407b1f-9376-5dd6-9cb1-3be59138a6bc
2025-12-05 20:02:22 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: d0407b1f-9376-5dd6-9cb1-3be59138a6bc, skipping ID generation
2025-12-05 20:02:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item d0407b1f-9376-5dd6-9cb1-3be59138a6bc with scrape type 1
2025-12-05 20:02:22 [crawlers.middlewares.photo_download_middleware] INFO: Removing item d0407b1f-9376-5dd6-9cb1-3be59138a6bc from tracking dictionaries
2025-12-05 20:02:22 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item d0407b1f-9376-5dd6-9cb1-3be59138a6bc sending to next pipeline
2025-12-05 20:02:22 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: d0407b1f-9376-5dd6-9cb1-3be59138a6bc, identifier: 1788856
2025-12-05 20:02:22 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:22 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:22 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:22 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0ae4790>
2025-12-05 20:02:22 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:22 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:23 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:02:23 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:23 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:23 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:23 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_130d34c1-ed5d-5c6b-b1f4-ea17b5728936/130d34c1-ed5d-5c6b-b1f4-ea17b5728936_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F383%252F0b821254a0f749fab0092d24858ebc5a%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/383/0b821254a0f749fab0092d24858ebc5a/images/large/1.jpg saved to cache
2025-12-05 20:02:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 130d34c1-ed5d-5c6b-b1f4-ea17b5728936
2025-12-05 20:02:23 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 130d34c1-ed5d-5c6b-b1f4-ea17b5728936, skipping ID generation
2025-12-05 20:02:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 130d34c1-ed5d-5c6b-b1f4-ea17b5728936 with scrape type 1
2025-12-05 20:02:23 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 130d34c1-ed5d-5c6b-b1f4-ea17b5728936 from tracking dictionaries
2025-12-05 20:02:23 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 130d34c1-ed5d-5c6b-b1f4-ea17b5728936 sending to next pipeline
2025-12-05 20:02:23 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 130d34c1-ed5d-5c6b-b1f4-ea17b5728936, identifier: 1788855
2025-12-05 20:02:23 [ayvens.es] INFO: Found listing with ID: 1788865
2025-12-05 20:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788865%27 HTTP/1.1" 200 None
2025-12-05 20:02:23 [ayvens.es] INFO: Scrape type for 1788865: 1
2025-12-05 20:02:23 [ayvens.es] INFO: Extracting data for car ID: 1788865
2025-12-05 20:02:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 9bdb369d-b46e-5f62-9ae1-f9acacab1ec0 with identifier: 1788865
2025-12-05 20:02:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9bdb369d-b46e-5f62-9ae1-f9acacab1ec0 with scrape type 1
2025-12-05 20:02:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 9bdb369d-b46e-5f62-9ae1-f9acacab1ec0
2025-12-05 20:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F820%252F49a4eb301b914ec697039e1ca43083f5%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/820/49a4eb301b914ec697039e1ca43083f5/images/large/1.jpg not downloaded yet
2025-12-05 20:02:23 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 43914037-7635-533e-bb58-b6c7e26a451a with identifier: 1788854
2025-12-05 20:02:23 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 43914037-7635-533e-bb58-b6c7e26a451a with identifier: 1788854
2025-12-05 20:02:23 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'citroën', 'model': 'c3', 'trim': 'puretech 60kw (83cv) feel', 'title': 'citroën c3 puretech 60kw (83cv) feel', 'price': 0, 'odometer': 57885, 'odometer_reading_unit': 'km', 'registration_date': '2020-01-10', 'year': 2020, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.2l', 'c_o_2_emission_value': 96, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '43914037-7635-533e-bb58-b6c7e26a451a', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_43914037-7635-533e-bb58-b6c7e26a451a/43914037-7635-533e-bb58-b6c7e26a451a_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/482/3ad3209cbaf94b6d8302d09bdd21b930/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 582489}], 'current_year_diff': 5, 'age_in_months': 70, 'make_model': 'citroën c3', 'sanitized_make': 'citron', 'odometer_in_km': 57885}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 70,
     'auction_title': '',
     'c_o_2_emission_value': 96,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.2l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': '43914037-7635-533e-bb58-b6c7e26a451a',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'citroën',
     'make_model': 'citroën c3',
     'model': 'c3',
     'odometer': 57885,
     'odometer_in_km': 57885,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_43914037-7635-533e-bb58-b6c7e26a451a/43914037-7635-533e-bb58-b6c7e26a451a_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 582489,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/482/3ad3209cbaf94b6d8302d09bdd21b930/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-01-10',
     'sanitized_make': 'citron',
     'seats_number': 5,
     'seller_name': None,
     'title': 'citroën c3 puretech 60kw (83cv) feel',
     'transmission': 'manual',
     'trim': 'puretech 60kw (83cv) feel',
     'vin': None,
     'year': 2020}
2025-12-05 20:02:23 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: PURETECH_FOR_COUNTRY failed for item: 43914037-7635-533e-bb58-b6c7e26a451a spider: ayvens.es
2025-12-05 20:02:23 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 43914037-7635-533e-bb58-b6c7e26a451a due to rule: puretech_for_country.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'citroën', 'model': 'c3', 'trim': 'puretech 60kw (83cv) feel', 'title': 'citroën c3 puretech 60kw (83cv) feel', 'price': 0, 'odometer': 57885, 'odometer_reading_unit': 'km', 'registration_date': '2020-01-10', 'year': 2020, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.2l', 'c_o_2_emission_value': 96, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '43914037-7635-533e-bb58-b6c7e26a451a', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_43914037-7635-533e-bb58-b6c7e26a451a/43914037-7635-533e-bb58-b6c7e26a451a_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/482/3ad3209cbaf94b6d8302d09bdd21b930/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 582489}], 'current_year_diff': 5, 'age_in_months': 70, 'make_model': 'citroën c3', 'sanitized_make': 'citron', 'odometer_in_km': 57885}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 70,
     'auction_title': '',
     'c_o_2_emission_value': 96,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.2l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': '43914037-7635-533e-bb58-b6c7e26a451a',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'citroën',
     'make_model': 'citroën c3',
     'model': 'c3',
     'odometer': 57885,
     'odometer_in_km': 57885,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_43914037-7635-533e-bb58-b6c7e26a451a/43914037-7635-533e-bb58-b6c7e26a451a_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 582489,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/482/3ad3209cbaf94b6d8302d09bdd21b930/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-01-10',
     'sanitized_make': 'citron',
     'seats_number': 5,
     'seller_name': None,
     'title': 'citroën c3 puretech 60kw (83cv) feel',
     'transmission': 'manual',
     'trim': 'puretech 60kw (83cv) feel',
     'vin': None,
     'year': 2020}
{'auction_title': '',
 'c_o_2_emission_value': 96,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.2L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '43914037-7635-533e-bb58-b6c7e26a451a',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'CITROËN',
 'model': 'C3',
 'odometer': 57885,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_43914037-7635-533e-bb58-b6c7e26a451a/43914037-7635-533e-bb58-b6c7e26a451a_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 582489,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/482/3ad3209cbaf94b6d8302d09bdd21b930/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-01-10',
 'seats_number': 5,
 'seller_name': None,
 'title': 'CITROËN C3 PureTech 60KW (83CV) FEEL',
 'transmission': 'manual',
 'trim': 'PureTech 60KW (83CV) FEEL',
 'vin': None,
 'year': 2020}
2025-12-05 20:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788854%27 HTTP/1.1" 200 None
2025-12-05 20:02:23 [ayvens.es] INFO: Saving data for 1788854: {'created_time': 1764964943.692157, 'last_price_update_time': 1764964943.692168, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788854') HTTP/1.1" 204 0
2025-12-05 20:02:23 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: d0407b1f-9376-5dd6-9cb1-3be59138a6bc with identifier: 1788856
2025-12-05 20:02:23 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: d0407b1f-9376-5dd6-9cb1-3be59138a6bc with identifier: 1788856
2025-12-05 20:02:23 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: 149118 is greater than the maximum of 130000

Failed validating 'maximum' in schema['allOf'][0]['then']['properties']['odometer_in_km']:
    {'maximum': 130000, 'type': 'integer'}

On instance['odometer_in_km']:
    149118
2025-12-05 20:02:23 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE_FOR_MAKE failed for item: d0407b1f-9376-5dd6-9cb1-3be59138a6bc spider: ayvens.es
2025-12-05 20:02:23 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: d0407b1f-9376-5dd6-9cb1-3be59138a6bc due to rule: high_mileage_for_make.json, error: 149118 is greater than the maximum of 130000

Failed validating 'maximum' in schema['allOf'][0]['then']['properties']['odometer_in_km']:
    {'maximum': 130000, 'type': 'integer'}

On instance['odometer_in_km']:
    149118
{'auction_title': '',
 'c_o_2_emission_value': 101,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'd0407b1f-9376-5dd6-9cb1-3be59138a6bc',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'MEGANE',
 'odometer': 149118,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_d0407b1f-9376-5dd6-9cb1-3be59138a6bc/d0407b1f-9376-5dd6-9cb1-3be59138a6bc_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 346996,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/168/6bec3f99eec64852857e7af6e2114b18/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-02-25',
 'seats_number': 5,
 'seller_name': None,
 'title': 'RENAULT MEGANE Business Blue dCi 85kW (115CV) - 18',
 'transmission': 'manual',
 'trim': 'Business Blue dCi 85kW (115CV) - 18',
 'vin': None,
 'year': 2019}
2025-12-05 20:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788856%27 HTTP/1.1" 200 None
2025-12-05 20:02:23 [ayvens.es] INFO: Saving data for 1788856: {'created_time': 1764964943.958674, 'last_price_update_time': 1764964943.958686, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788856') HTTP/1.1" 204 0
2025-12-05 20:02:23 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 130d34c1-ed5d-5c6b-b1f4-ea17b5728936 with identifier: 1788855
2025-12-05 20:02:23 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 130d34c1-ed5d-5c6b-b1f4-ea17b5728936 with identifier: 1788855
2025-12-05 20:02:24 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 130d34c1-ed5d-5c6b-b1f4-ea17b5728936 with identifier: 1788855 to the API
2025-12-05 20:02:24 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:24 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:24 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/383/0b821254a0f749fab0092d24858ebc5a/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 137,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '130d34c1-ed5d-5c6b-b1f4-ea17b5728936',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'DS AUTOMOBILES',
 'model': 'DS 7 CROSSBACK',
 'odometer': 49494,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_130d34c1-ed5d-5c6b-b1f4-ea17b5728936/130d34c1-ed5d-5c6b-b1f4-ea17b5728936_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 343414,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/383/0b821254a0f749fab0092d24858ebc5a/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-12-20',
 'seats_number': 5,
 'seller_name': None,
 'title': 'DS AUTOMOBILES DS 7 CROSSBACK BlueHDi DE 96kW (130CV) AT Bastille +',
 'transmission': 'manual',
 'trim': 'BlueHDi DE 96kW (130CV) AT Bastille +',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788855%27 HTTP/1.1" 200 None
2025-12-05 20:02:24 [ayvens.es] INFO: Saving data for 1788855: {'created_time': 1764964944.330002, 'last_price_update_time': 1764964944.330011, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788855') HTTP/1.1" 204 0
2025-12-05 20:02:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/398/955c51bad48e4157bd5035b12d973999/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/950/55c8537402424169b7df219b2f7e5ab2/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:24 [ayvens.es] INFO: Found listing with ID: 1788866
2025-12-05 20:02:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788866%27 HTTP/1.1" 200 None
2025-12-05 20:02:24 [ayvens.es] INFO: Scrape type for 1788866: 1
2025-12-05 20:02:24 [ayvens.es] INFO: Extracting data for car ID: 1788866
2025-12-05 20:02:24 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 74ac6544-42e5-5bab-995a-813dd5704d95 with identifier: 1788866
2025-12-05 20:02:24 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 74ac6544-42e5-5bab-995a-813dd5704d95 with scrape type 1
2025-12-05 20:02:24 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 74ac6544-42e5-5bab-995a-813dd5704d95
2025-12-05 20:02:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F725%252F2bb95cb1d8fc4d7c8e58d9991cee745f%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/725/2bb95cb1d8fc4d7c8e58d9991cee745f/images/large/1.jpg not downloaded yet
2025-12-05 20:02:24 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:24 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:24 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:24 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b83150>
2025-12-05 20:02:24 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:24 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:24 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1971
2025-12-05 20:02:24 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:24 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:24 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:24 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_647645f7-9623-52d5-8508-a7fec369d75d/647645f7-9623-52d5-8508-a7fec369d75d_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F533%252F70ee3ebeb2bb4706a6a34a3a34b709a2%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/533/70ee3ebeb2bb4706a6a34a3a34b709a2/images/large/1.jpg saved to cache
2025-12-05 20:02:24 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 647645f7-9623-52d5-8508-a7fec369d75d
2025-12-05 20:02:24 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 647645f7-9623-52d5-8508-a7fec369d75d, skipping ID generation
2025-12-05 20:02:24 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 647645f7-9623-52d5-8508-a7fec369d75d with scrape type 1
2025-12-05 20:02:24 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 647645f7-9623-52d5-8508-a7fec369d75d from tracking dictionaries
2025-12-05 20:02:24 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 647645f7-9623-52d5-8508-a7fec369d75d sending to next pipeline
2025-12-05 20:02:24 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 647645f7-9623-52d5-8508-a7fec369d75d, identifier: 1788857
2025-12-05 20:02:24 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:24 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:24 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:24 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0472b50>
2025-12-05 20:02:24 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:24 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:25 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1969
2025-12-05 20:02:25 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:25 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:25 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:25 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_43e1ac88-e2f6-57c4-8e1f-3a361adf1655/43e1ac88-e2f6-57c4-8e1f-3a361adf1655_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F926%252F98ef3930cfc54214ad0f470c17dd29a5%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/926/98ef3930cfc54214ad0f470c17dd29a5/images/large/1.jpg saved to cache
2025-12-05 20:02:25 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 43e1ac88-e2f6-57c4-8e1f-3a361adf1655
2025-12-05 20:02:25 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 43e1ac88-e2f6-57c4-8e1f-3a361adf1655, skipping ID generation
2025-12-05 20:02:25 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 43e1ac88-e2f6-57c4-8e1f-3a361adf1655 with scrape type 1
2025-12-05 20:02:25 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 43e1ac88-e2f6-57c4-8e1f-3a361adf1655 from tracking dictionaries
2025-12-05 20:02:25 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 43e1ac88-e2f6-57c4-8e1f-3a361adf1655 sending to next pipeline
2025-12-05 20:02:25 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 43e1ac88-e2f6-57c4-8e1f-3a361adf1655, identifier: 1788858
2025-12-05 20:02:25 [ayvens.es] INFO: Found listing with ID: 1788867
2025-12-05 20:02:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788867%27 HTTP/1.1" 200 None
2025-12-05 20:02:25 [ayvens.es] INFO: Scrape type for 1788867: 1
2025-12-05 20:02:25 [ayvens.es] INFO: Extracting data for car ID: 1788867
2025-12-05 20:02:25 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: d8e6f5f6-fa7e-5c6d-91a2-351921d13f93 with identifier: 1788867
2025-12-05 20:02:25 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item d8e6f5f6-fa7e-5c6d-91a2-351921d13f93 with scrape type 1
2025-12-05 20:02:25 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item d8e6f5f6-fa7e-5c6d-91a2-351921d13f93
2025-12-05 20:02:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F773%252F472e1714abac402bae8007ee0df0de76%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/773/472e1714abac402bae8007ee0df0de76/images/large/1.jpg not downloaded yet
2025-12-05 20:02:25 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 647645f7-9623-52d5-8508-a7fec369d75d with identifier: 1788857
2025-12-05 20:02:25 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 647645f7-9623-52d5-8508-a7fec369d75d with identifier: 1788857
2025-12-05 20:02:25 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 647645f7-9623-52d5-8508-a7fec369d75d with identifier: 1788857 to the API
2025-12-05 20:02:25 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:25 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:25 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/533/70ee3ebeb2bb4706a6a34a3a34b709a2/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 134,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '647645f7-9623-52d5-8508-a7fec369d75d',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'NISSAN',
 'model': 'JUKE',
 'odometer': 48651,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_647645f7-9623-52d5-8508-a7fec369d75d/647645f7-9623-52d5-8508-a7fec369d75d_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 297725,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/533/70ee3ebeb2bb4706a6a34a3a34b709a2/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-11-12',
 'seats_number': 5,
 'seller_name': None,
 'title': 'NISSAN JUKE DIG-T 84 kW (114 CV) 6M/T Acenta',
 'transmission': 'manual',
 'trim': 'DIG-T 84 kW (114 CV) 6M/T Acenta',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788857%27 HTTP/1.1" 200 None
2025-12-05 20:02:25 [ayvens.es] INFO: Saving data for 1788857: {'created_time': 1764964945.654609, 'last_price_update_time': 1764964945.654619, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788857') HTTP/1.1" 204 0
2025-12-05 20:02:25 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 43e1ac88-e2f6-57c4-8e1f-3a361adf1655 with identifier: 1788858
2025-12-05 20:02:25 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 43e1ac88-e2f6-57c4-8e1f-3a361adf1655 with identifier: 1788858
2025-12-05 20:02:25 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'peugeot', 'model': '2008', 'trim': 'active puretech 100 s&s bvm6', 'title': 'peugeot 2008 active puretech 100 s&s bvm6', 'price': 0, 'odometer': 81698, 'odometer_reading_unit': 'km', 'registration_date': '2020-11-23', 'year': 2020, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.2l', 'c_o_2_emission_value': 122, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '43e1ac88-e2f6-57c4-8e1f-3a361adf1655', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_43e1ac88-e2f6-57c4-8e1f-3a361adf1655/43e1ac88-e2f6-57c4-8e1f-3a361adf1655_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/926/98ef3930cfc54214ad0f470c17dd29a5/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 353636}], 'current_year_diff': 5, 'age_in_months': 60, 'make_model': 'peugeot 2008', 'sanitized_make': 'peugeot', 'odometer_in_km': 81698}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 60,
     'auction_title': '',
     'c_o_2_emission_value': 122,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.2l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': '43e1ac88-e2f6-57c4-8e1f-3a361adf1655',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'peugeot',
     'make_model': 'peugeot 2008',
     'model': '2008',
     'odometer': 81698,
     'odometer_in_km': 81698,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_43e1ac88-e2f6-57c4-8e1f-3a361adf1655/43e1ac88-e2f6-57c4-8e1f-3a361adf1655_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 353636,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/926/98ef3930cfc54214ad0f470c17dd29a5/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-11-23',
     'sanitized_make': 'peugeot',
     'seats_number': 5,
     'seller_name': None,
     'title': 'peugeot 2008 active puretech 100 s&s bvm6',
     'transmission': 'manual',
     'trim': 'active puretech 100 s&s bvm6',
     'vin': None,
     'year': 2020}
2025-12-05 20:02:25 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: PURETECH_FOR_COUNTRY failed for item: 43e1ac88-e2f6-57c4-8e1f-3a361adf1655 spider: ayvens.es
2025-12-05 20:02:25 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 43e1ac88-e2f6-57c4-8e1f-3a361adf1655 due to rule: puretech_for_country.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'peugeot', 'model': '2008', 'trim': 'active puretech 100 s&s bvm6', 'title': 'peugeot 2008 active puretech 100 s&s bvm6', 'price': 0, 'odometer': 81698, 'odometer_reading_unit': 'km', 'registration_date': '2020-11-23', 'year': 2020, 'fuel_type': 'petrol', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.2l', 'c_o_2_emission_value': 122, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '43e1ac88-e2f6-57c4-8e1f-3a361adf1655', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_43e1ac88-e2f6-57c4-8e1f-3a361adf1655/43e1ac88-e2f6-57c4-8e1f-3a361adf1655_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/926/98ef3930cfc54214ad0f470c17dd29a5/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 353636}], 'current_year_diff': 5, 'age_in_months': 60, 'make_model': 'peugeot 2008', 'sanitized_make': 'peugeot', 'odometer_in_km': 81698}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 60,
     'auction_title': '',
     'c_o_2_emission_value': 122,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.2l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'petrol',
     'id': '43e1ac88-e2f6-57c4-8e1f-3a361adf1655',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'peugeot',
     'make_model': 'peugeot 2008',
     'model': '2008',
     'odometer': 81698,
     'odometer_in_km': 81698,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_43e1ac88-e2f6-57c4-8e1f-3a361adf1655/43e1ac88-e2f6-57c4-8e1f-3a361adf1655_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 353636,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/926/98ef3930cfc54214ad0f470c17dd29a5/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-11-23',
     'sanitized_make': 'peugeot',
     'seats_number': 5,
     'seller_name': None,
     'title': 'peugeot 2008 active puretech 100 s&s bvm6',
     'transmission': 'manual',
     'trim': 'active puretech 100 s&s bvm6',
     'vin': None,
     'year': 2020}
{'auction_title': '',
 'c_o_2_emission_value': 122,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.2L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '43e1ac88-e2f6-57c4-8e1f-3a361adf1655',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': '2008',
 'odometer': 81698,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_43e1ac88-e2f6-57c4-8e1f-3a361adf1655/43e1ac88-e2f6-57c4-8e1f-3a361adf1655_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 353636,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/926/98ef3930cfc54214ad0f470c17dd29a5/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-11-23',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT 2008 Active Puretech 100 S&S BVM6',
 'transmission': 'manual',
 'trim': 'Active Puretech 100 S&S BVM6',
 'vin': None,
 'year': 2020}
2025-12-05 20:02:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788858%27 HTTP/1.1" 200 None
2025-12-05 20:02:25 [ayvens.es] INFO: Saving data for 1788858: {'created_time': 1764964945.941513, 'last_price_update_time': 1764964945.941524, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788858') HTTP/1.1" 204 0
2025-12-05 20:02:25 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/360/5e72d1785c2741a88879d1cbc48c7006/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:25 [ayvens.es] INFO: Found listing with ID: 1788868
2025-12-05 20:02:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788868%27 HTTP/1.1" 200 None
2025-12-05 20:02:25 [ayvens.es] INFO: Scrape type for 1788868: 1
2025-12-05 20:02:25 [ayvens.es] INFO: Extracting data for car ID: 1788868
2025-12-05 20:02:25 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: e3226609-95c1-50fe-a295-d8ca46440a6f with identifier: 1788868
2025-12-05 20:02:25 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e3226609-95c1-50fe-a295-d8ca46440a6f with scrape type 1
2025-12-05 20:02:25 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item e3226609-95c1-50fe-a295-d8ca46440a6f
2025-12-05 20:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F879%252F4c9260dff41b43ecbd2162eb1dfa6124%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/879/4c9260dff41b43ecbd2162eb1dfa6124/images/large/1.jpg not downloaded yet
2025-12-05 20:02:26 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:26 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:26 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:26 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b0f610>
2025-12-05 20:02:26 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:26 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:26 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:02:26 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:26 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:26 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:26 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_5d32e842-4e5b-5de8-840d-5b438ec73f75/5d32e842-4e5b-5de8-840d-5b438ec73f75_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F398%252F955c51bad48e4157bd5035b12d973999%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/398/955c51bad48e4157bd5035b12d973999/images/large/1.jpg saved to cache
2025-12-05 20:02:26 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 5d32e842-4e5b-5de8-840d-5b438ec73f75
2025-12-05 20:02:26 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 5d32e842-4e5b-5de8-840d-5b438ec73f75, skipping ID generation
2025-12-05 20:02:26 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 5d32e842-4e5b-5de8-840d-5b438ec73f75 with scrape type 1
2025-12-05 20:02:26 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 5d32e842-4e5b-5de8-840d-5b438ec73f75 from tracking dictionaries
2025-12-05 20:02:26 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 5d32e842-4e5b-5de8-840d-5b438ec73f75 sending to next pipeline
2025-12-05 20:02:26 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 5d32e842-4e5b-5de8-840d-5b438ec73f75, identifier: 1788859
2025-12-05 20:02:26 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:26 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:26 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:26 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0527350>
2025-12-05 20:02:26 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:26 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:26 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:02:26 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:26 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:26 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:26 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_b795cc83-6a67-5cf7-a253-d0215447f3ef/b795cc83-6a67-5cf7-a253-d0215447f3ef_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F950%252F55c8537402424169b7df219b2f7e5ab2%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/950/55c8537402424169b7df219b2f7e5ab2/images/large/1.jpg saved to cache
2025-12-05 20:02:26 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item b795cc83-6a67-5cf7-a253-d0215447f3ef
2025-12-05 20:02:26 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: b795cc83-6a67-5cf7-a253-d0215447f3ef, skipping ID generation
2025-12-05 20:02:26 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b795cc83-6a67-5cf7-a253-d0215447f3ef with scrape type 1
2025-12-05 20:02:26 [crawlers.middlewares.photo_download_middleware] INFO: Removing item b795cc83-6a67-5cf7-a253-d0215447f3ef from tracking dictionaries
2025-12-05 20:02:26 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item b795cc83-6a67-5cf7-a253-d0215447f3ef sending to next pipeline
2025-12-05 20:02:26 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: b795cc83-6a67-5cf7-a253-d0215447f3ef, identifier: 1788860
2025-12-05 20:02:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/887/a1e83022113e4852ad1d71a63b613221/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:26 [ayvens.es] INFO: Found listing with ID: 1788869
2025-12-05 20:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788869%27 HTTP/1.1" 200 None
2025-12-05 20:02:26 [ayvens.es] INFO: Scrape type for 1788869: 1
2025-12-05 20:02:26 [ayvens.es] INFO: Extracting data for car ID: 1788869
2025-12-05 20:02:26 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 66348a8e-f41f-5373-a724-6b5b4f0bfa77 with identifier: 1788869
2025-12-05 20:02:26 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 66348a8e-f41f-5373-a724-6b5b4f0bfa77 with scrape type 1
2025-12-05 20:02:26 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 66348a8e-f41f-5373-a724-6b5b4f0bfa77
2025-12-05 20:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F685%252F2580bca87c994d1a9b7e69571dcecd7b%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/685/2580bca87c994d1a9b7e69571dcecd7b/images/large/1.jpg not downloaded yet
2025-12-05 20:02:26 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 5d32e842-4e5b-5de8-840d-5b438ec73f75 with identifier: 1788859
2025-12-05 20:02:26 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 5d32e842-4e5b-5de8-840d-5b438ec73f75 with identifier: 1788859
2025-12-05 20:02:27 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'volkswagen', 'model': 'caddy', 'trim': 'profesional kombi 2.0 tdi 90kw bmt 4mot', 'title': 'volkswagen caddy profesional kombi 2.0 tdi 90kw bmt 4mot', 'price': 0, 'odometer': 222877, 'odometer_reading_unit': 'km', 'registration_date': '2017-12-13', 'year': 2017, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 4, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '2.0l', 'c_o_2_emission_value': 150, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '5d32e842-4e5b-5de8-840d-5b438ec73f75', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_5d32e842-4e5b-5de8-840d-5b438ec73f75/5d32e842-4e5b-5de8-840d-5b438ec73f75_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/398/955c51bad48e4157bd5035b12d973999/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 342082}], 'current_year_diff': 8, 'age_in_months': 95, 'make_model': 'volkswagen caddy', 'sanitized_make': 'volkswagen', 'odometer_in_km': 222877}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 95,
     'auction_title': '',
     'c_o_2_emission_value': 150,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 8,
     'doors_number': 4,
     'engine': '2.0l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': '5d32e842-4e5b-5de8-840d-5b438ec73f75',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'volkswagen',
     'make_model': 'volkswagen caddy',
     'model': 'caddy',
     'odometer': 222877,
     'odometer_in_km': 222877,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_5d32e842-4e5b-5de8-840d-5b438ec73f75/5d32e842-4e5b-5de8-840d-5b438ec73f75_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 342082,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/398/955c51bad48e4157bd5035b12d973999/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2017-12-13',
     'sanitized_make': 'volkswagen',
     'seats_number': 5,
     'seller_name': None,
     'title': 'volkswagen caddy profesional kombi 2.0 tdi 90kw bmt 4mot',
     'transmission': 'manual',
     'trim': 'profesional kombi 2.0 tdi 90kw bmt 4mot',
     'vin': None,
     'year': 2017}
2025-12-05 20:02:27 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE failed for item: 5d32e842-4e5b-5de8-840d-5b438ec73f75 spider: ayvens.es
2025-12-05 20:02:27 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 5d32e842-4e5b-5de8-840d-5b438ec73f75 due to rule: high_mileage.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'volkswagen', 'model': 'caddy', 'trim': 'profesional kombi 2.0 tdi 90kw bmt 4mot', 'title': 'volkswagen caddy profesional kombi 2.0 tdi 90kw bmt 4mot', 'price': 0, 'odometer': 222877, 'odometer_reading_unit': 'km', 'registration_date': '2017-12-13', 'year': 2017, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 4, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '2.0l', 'c_o_2_emission_value': 150, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '5d32e842-4e5b-5de8-840d-5b438ec73f75', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_5d32e842-4e5b-5de8-840d-5b438ec73f75/5d32e842-4e5b-5de8-840d-5b438ec73f75_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/398/955c51bad48e4157bd5035b12d973999/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 342082}], 'current_year_diff': 8, 'age_in_months': 95, 'make_model': 'volkswagen caddy', 'sanitized_make': 'volkswagen', 'odometer_in_km': 222877}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 95,
     'auction_title': '',
     'c_o_2_emission_value': 150,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 8,
     'doors_number': 4,
     'engine': '2.0l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': '5d32e842-4e5b-5de8-840d-5b438ec73f75',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'volkswagen',
     'make_model': 'volkswagen caddy',
     'model': 'caddy',
     'odometer': 222877,
     'odometer_in_km': 222877,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_5d32e842-4e5b-5de8-840d-5b438ec73f75/5d32e842-4e5b-5de8-840d-5b438ec73f75_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 342082,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/398/955c51bad48e4157bd5035b12d973999/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2017-12-13',
     'sanitized_make': 'volkswagen',
     'seats_number': 5,
     'seller_name': None,
     'title': 'volkswagen caddy profesional kombi 2.0 tdi 90kw bmt 4mot',
     'transmission': 'manual',
     'trim': 'profesional kombi 2.0 tdi 90kw bmt 4mot',
     'vin': None,
     'year': 2017}
{'auction_title': '',
 'c_o_2_emission_value': 150,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '5d32e842-4e5b-5de8-840d-5b438ec73f75',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'CADDY',
 'odometer': 222877,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_5d32e842-4e5b-5de8-840d-5b438ec73f75/5d32e842-4e5b-5de8-840d-5b438ec73f75_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 342082,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/398/955c51bad48e4157bd5035b12d973999/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2017-12-13',
 'seats_number': 5,
 'seller_name': None,
 'title': 'VOLKSWAGEN CADDY Profesional Kombi 2.0 TDI 90kW BMT 4Mot',
 'transmission': 'manual',
 'trim': 'Profesional Kombi 2.0 TDI 90kW BMT 4Mot',
 'vin': None,
 'year': 2017}
2025-12-05 20:02:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788859%27 HTTP/1.1" 200 None
2025-12-05 20:02:27 [ayvens.es] INFO: Saving data for 1788859: {'created_time': 1764964947.168792, 'last_price_update_time': 1764964947.168804, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788859') HTTP/1.1" 204 0
2025-12-05 20:02:27 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: b795cc83-6a67-5cf7-a253-d0215447f3ef with identifier: 1788860
2025-12-05 20:02:27 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: b795cc83-6a67-5cf7-a253-d0215447f3ef with identifier: 1788860
2025-12-05 20:02:27 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: b795cc83-6a67-5cf7-a253-d0215447f3ef with identifier: 1788860 to the API
2025-12-05 20:02:27 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:27 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:27 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/950/55c8537402424169b7df219b2f7e5ab2/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 111,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.6L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'b795cc83-6a67-5cf7-a253-d0215447f3ef',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'T-ROC',
 'odometer': 74584,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_b795cc83-6a67-5cf7-a253-d0215447f3ef/b795cc83-6a67-5cf7-a253-d0215447f3ef_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 361782,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/950/55c8537402424169b7df219b2f7e5ab2/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-11-11',
 'seats_number': 5,
 'seller_name': None,
 'title': 'VOLKSWAGEN T-ROC Edition 1.6 TDI 85kW (115CV)',
 'transmission': 'manual',
 'trim': 'Edition 1.6 TDI 85kW (115CV)',
 'vin': None,
 'year': 2020}
2025-12-05 20:02:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788860%27 HTTP/1.1" 200 None
2025-12-05 20:02:27 [ayvens.es] INFO: Saving data for 1788860: {'created_time': 1764964947.642063, 'last_price_update_time': 1764964947.642072, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788860') HTTP/1.1" 204 0
2025-12-05 20:02:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/240/5e5bf5b5f2284a1faf5a46071c78b58c/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:27 [ayvens.es] INFO: Found listing with ID: 1788870
2025-12-05 20:02:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788870%27 HTTP/1.1" 200 None
2025-12-05 20:02:27 [ayvens.es] INFO: Scrape type for 1788870: 1
2025-12-05 20:02:27 [ayvens.es] INFO: Extracting data for car ID: 1788870
2025-12-05 20:02:27 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 3ab59d01-2d64-5645-831e-54378e0c3941 with identifier: 1788870
2025-12-05 20:02:27 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 3ab59d01-2d64-5645-831e-54378e0c3941 with scrape type 1
2025-12-05 20:02:27 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 3ab59d01-2d64-5645-831e-54378e0c3941
2025-12-05 20:02:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F690%252F04126256354a4c998930421bbfbfab0f%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/690/04126256354a4c998930421bbfbfab0f/images/large/1.jpg not downloaded yet
2025-12-05 20:02:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/279/55c3f0f83ba64b29a525c7d6872d355e/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:27 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:27 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:27 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:27 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0ae7050>
2025-12-05 20:02:27 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:27 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:27 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:02:27 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:27 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:27 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:28 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_789dfe9a-fc34-5022-bb66-ec7da309f4c9/789dfe9a-fc34-5022-bb66-ec7da309f4c9_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F360%252F5e72d1785c2741a88879d1cbc48c7006%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/360/5e72d1785c2741a88879d1cbc48c7006/images/large/1.jpg saved to cache
2025-12-05 20:02:28 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 789dfe9a-fc34-5022-bb66-ec7da309f4c9
2025-12-05 20:02:28 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 789dfe9a-fc34-5022-bb66-ec7da309f4c9, skipping ID generation
2025-12-05 20:02:28 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 789dfe9a-fc34-5022-bb66-ec7da309f4c9 with scrape type 1
2025-12-05 20:02:28 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 789dfe9a-fc34-5022-bb66-ec7da309f4c9 from tracking dictionaries
2025-12-05 20:02:28 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 789dfe9a-fc34-5022-bb66-ec7da309f4c9 sending to next pipeline
2025-12-05 20:02:28 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 789dfe9a-fc34-5022-bb66-ec7da309f4c9, identifier: 1788862
2025-12-05 20:02:28 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:28 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:28 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:28 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b3ec0c90>
2025-12-05 20:02:28 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:28 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:28 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:02:28 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:28 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:28 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:28 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_38d68140-8304-5b0f-9632-11d274139ca3/38d68140-8304-5b0f-9632-11d274139ca3_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F887%252Fa1e83022113e4852ad1d71a63b613221%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/887/a1e83022113e4852ad1d71a63b613221/images/large/1.jpg saved to cache
2025-12-05 20:02:28 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 38d68140-8304-5b0f-9632-11d274139ca3
2025-12-05 20:02:28 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 38d68140-8304-5b0f-9632-11d274139ca3, skipping ID generation
2025-12-05 20:02:28 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 38d68140-8304-5b0f-9632-11d274139ca3 with scrape type 1
2025-12-05 20:02:28 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 38d68140-8304-5b0f-9632-11d274139ca3 from tracking dictionaries
2025-12-05 20:02:28 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 38d68140-8304-5b0f-9632-11d274139ca3 sending to next pipeline
2025-12-05 20:02:28 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 38d68140-8304-5b0f-9632-11d274139ca3, identifier: 1788861
2025-12-05 20:02:28 [ayvens.es] INFO: Found listing with ID: 1788871
2025-12-05 20:02:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788871%27 HTTP/1.1" 200 None
2025-12-05 20:02:28 [ayvens.es] INFO: Scrape type for 1788871: 1
2025-12-05 20:02:28 [ayvens.es] INFO: Extracting data for car ID: 1788871
2025-12-05 20:02:28 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: fdc12952-17a4-5be1-a5d9-c7988ff18dbe with identifier: 1788871
2025-12-05 20:02:28 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item fdc12952-17a4-5be1-a5d9-c7988ff18dbe with scrape type 1
2025-12-05 20:02:28 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item fdc12952-17a4-5be1-a5d9-c7988ff18dbe
2025-12-05 20:02:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F352%252F65241b5880784f65af6b7d97e3350640%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/352/65241b5880784f65af6b7d97e3350640/images/large/1.jpg not downloaded yet
2025-12-05 20:02:28 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 789dfe9a-fc34-5022-bb66-ec7da309f4c9 with identifier: 1788862
2025-12-05 20:02:28 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 789dfe9a-fc34-5022-bb66-ec7da309f4c9 with identifier: 1788862
2025-12-05 20:02:28 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 789dfe9a-fc34-5022-bb66-ec7da309f4c9 with identifier: 1788862 to the API
2025-12-05 20:02:28 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:28 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:29 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/360/5e72d1785c2741a88879d1cbc48c7006/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 119,
 'color': 'Azul',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '789dfe9a-fc34-5022-bb66-ec7da309f4c9',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'FORD',
 'model': 'PUMA',
 'odometer': 17999,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_789dfe9a-fc34-5022-bb66-ec7da309f4c9/789dfe9a-fc34-5022-bb66-ec7da309f4c9_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 386951,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/360/5e72d1785c2741a88879d1cbc48c7006/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-08-18',
 'seats_number': 5,
 'seller_name': None,
 'title': 'FORD PUMA 1.5 Ecoblue 120cv Titanium',
 'transmission': 'manual',
 'trim': '1.5 Ecoblue 120cv Titanium',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788862%27 HTTP/1.1" 200 None
2025-12-05 20:02:29 [ayvens.es] INFO: Saving data for 1788862: {'created_time': 1764964949.058533, 'last_price_update_time': 1764964949.058543, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788862') HTTP/1.1" 204 0
2025-12-05 20:02:29 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 38d68140-8304-5b0f-9632-11d274139ca3 with identifier: 1788861
2025-12-05 20:02:29 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:02:29 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 38d68140-8304-5b0f-9632-11d274139ca3 with identifier: 1788861
2025-12-05 20:02:29 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 38d68140-8304-5b0f-9632-11d274139ca3 with identifier: 1788861 to the API
2025-12-05 20:02:29 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:29 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:29 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/887/a1e83022113e4852ad1d71a63b613221/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 120,
 'color': 'Alpinwei',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '38d68140-8304-5b0f-9632-11d274139ca3',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'BMW',
 'model': 'X1',
 'odometer': 85258,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_38d68140-8304-5b0f-9632-11d274139ca3/38d68140-8304-5b0f-9632-11d274139ca3_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 519852,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/887/a1e83022113e4852ad1d71a63b613221/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-05-03',
 'seats_number': 5,
 'seller_name': None,
 'title': 'BMW X1 sDrive18d',
 'transmission': 'manual',
 'trim': 'sDrive18d',
 'vin': None,
 'year': 2019}
2025-12-05 20:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788861%27 HTTP/1.1" 200 None
2025-12-05 20:02:29 [ayvens.es] INFO: Saving data for 1788861: {'created_time': 1764964949.612304, 'last_price_update_time': 1764964949.612313, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788861') HTTP/1.1" 204 0
2025-12-05 20:02:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/820/49a4eb301b914ec697039e1ca43083f5/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:29 [ayvens.es] INFO: Found listing with ID: 1788872
2025-12-05 20:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788872%27 HTTP/1.1" 200 None
2025-12-05 20:02:29 [ayvens.es] INFO: Scrape type for 1788872: 1
2025-12-05 20:02:29 [ayvens.es] INFO: Extracting data for car ID: 1788872
2025-12-05 20:02:29 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: be1fab5b-2d09-510b-b699-499311efdfab with identifier: 1788872
2025-12-05 20:02:29 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item be1fab5b-2d09-510b-b699-499311efdfab with scrape type 1
2025-12-05 20:02:29 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item be1fab5b-2d09-510b-b699-499311efdfab
2025-12-05 20:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F722%252F5e016041c81c460b831ab26221f49071%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/722/5e016041c81c460b831ab26221f49071/images/large/1.jpg not downloaded yet
2025-12-05 20:02:29 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:29 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:29 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:29 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0dc2b90>
2025-12-05 20:02:29 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:29 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:29 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:02:29 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:29 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:29 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:29 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_491b615f-2d91-561d-99e4-fb1c9ad45cc7/491b615f-2d91-561d-99e4-fb1c9ad45cc7_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F240%252F5e5bf5b5f2284a1faf5a46071c78b58c%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/240/5e5bf5b5f2284a1faf5a46071c78b58c/images/large/1.jpg saved to cache
2025-12-05 20:02:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 491b615f-2d91-561d-99e4-fb1c9ad45cc7
2025-12-05 20:02:30 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 491b615f-2d91-561d-99e4-fb1c9ad45cc7, skipping ID generation
2025-12-05 20:02:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 491b615f-2d91-561d-99e4-fb1c9ad45cc7 with scrape type 1
2025-12-05 20:02:30 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 491b615f-2d91-561d-99e4-fb1c9ad45cc7 from tracking dictionaries
2025-12-05 20:02:30 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 491b615f-2d91-561d-99e4-fb1c9ad45cc7 sending to next pipeline
2025-12-05 20:02:30 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 491b615f-2d91-561d-99e4-fb1c9ad45cc7, identifier: 1788864
2025-12-05 20:02:30 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:30 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:30 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:30 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b051f750>
2025-12-05 20:02:30 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:30 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:30 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:02:30 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:30 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:30 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:30 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_8ebf955d-a20c-5671-b328-40af0c866d5b/8ebf955d-a20c-5671-b328-40af0c866d5b_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F279%252F55c3f0f83ba64b29a525c7d6872d355e%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/279/55c3f0f83ba64b29a525c7d6872d355e/images/large/1.jpg saved to cache
2025-12-05 20:02:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 8ebf955d-a20c-5671-b328-40af0c866d5b
2025-12-05 20:02:30 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 8ebf955d-a20c-5671-b328-40af0c866d5b, skipping ID generation
2025-12-05 20:02:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 8ebf955d-a20c-5671-b328-40af0c866d5b with scrape type 1
2025-12-05 20:02:30 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 8ebf955d-a20c-5671-b328-40af0c866d5b from tracking dictionaries
2025-12-05 20:02:30 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 8ebf955d-a20c-5671-b328-40af0c866d5b sending to next pipeline
2025-12-05 20:02:30 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 8ebf955d-a20c-5671-b328-40af0c866d5b, identifier: 1788863
2025-12-05 20:02:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/725/2bb95cb1d8fc4d7c8e58d9991cee745f/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:30 [ayvens.es] INFO: Found listing with ID: 1788873
2025-12-05 20:02:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788873%27 HTTP/1.1" 200 None
2025-12-05 20:02:30 [ayvens.es] INFO: Scrape type for 1788873: 1
2025-12-05 20:02:30 [ayvens.es] INFO: Extracting data for car ID: 1788873
2025-12-05 20:02:30 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 6ac01f5c-c3dd-54c9-927c-7b8f74db3393 with identifier: 1788873
2025-12-05 20:02:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6ac01f5c-c3dd-54c9-927c-7b8f74db3393 with scrape type 1
2025-12-05 20:02:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 6ac01f5c-c3dd-54c9-927c-7b8f74db3393
2025-12-05 20:02:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F551%252Ffba99fad1f474c5496eb3ff911e59b95%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/551/fba99fad1f474c5496eb3ff911e59b95/images/large/1.jpg not downloaded yet
2025-12-05 20:02:30 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 491b615f-2d91-561d-99e4-fb1c9ad45cc7 with identifier: 1788864
2025-12-05 20:02:30 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 491b615f-2d91-561d-99e4-fb1c9ad45cc7 with identifier: 1788864
2025-12-05 20:02:30 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 491b615f-2d91-561d-99e4-fb1c9ad45cc7 with identifier: 1788864 to the API
2025-12-05 20:02:30 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:30 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:31 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/240/5e5bf5b5f2284a1faf5a46071c78b58c/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 102,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.6L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '491b615f-2d91-561d-99e4-fb1c9ad45cc7',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'GOLF',
 'odometer': 81481,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_491b615f-2d91-561d-99e4-fb1c9ad45cc7/491b615f-2d91-561d-99e4-fb1c9ad45cc7_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 364289,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/240/5e5bf5b5f2284a1faf5a46071c78b58c/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2018-10-17',
 'seats_number': 5,
 'seller_name': None,
 'title': 'VOLKSWAGEN GOLF Advance 1.6 TDI 85kW (115CV) DSG',
 'transmission': 'manual',
 'trim': 'Advance 1.6 TDI 85kW (115CV) DSG',
 'vin': None,
 'year': 2018}
2025-12-05 20:02:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788864%27 HTTP/1.1" 200 None
2025-12-05 20:02:31 [ayvens.es] INFO: Saving data for 1788864: {'created_time': 1764964951.019862, 'last_price_update_time': 1764964951.019873, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788864') HTTP/1.1" 204 0
2025-12-05 20:02:31 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 8ebf955d-a20c-5671-b328-40af0c866d5b with identifier: 1788863
2025-12-05 20:02:31 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 8ebf955d-a20c-5671-b328-40af0c866d5b with identifier: 1788863
2025-12-05 20:02:31 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 8ebf955d-a20c-5671-b328-40af0c866d5b with identifier: 1788863 to the API
2025-12-05 20:02:31 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:31 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:31 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/279/55c3f0f83ba64b29a525c7d6872d355e/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 134,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '8ebf955d-a20c-5671-b328-40af0c866d5b',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'NISSAN',
 'model': 'JUKE',
 'odometer': 37795,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_8ebf955d-a20c-5671-b328-40af0c866d5b/8ebf955d-a20c-5671-b328-40af0c866d5b_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 426814,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/279/55c3f0f83ba64b29a525c7d6872d355e/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-08-30',
 'seats_number': 5,
 'seller_name': None,
 'title': 'NISSAN JUKE DIG-T 84 kW (114 CV) 6M/T N-Connecta',
 'transmission': 'manual',
 'trim': 'DIG-T 84 kW (114 CV) 6M/T N-Connecta',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788863%27 HTTP/1.1" 200 None
2025-12-05 20:02:31 [ayvens.es] INFO: Saving data for 1788863: {'created_time': 1764964951.701154, 'last_price_update_time': 1764964951.701165, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788863') HTTP/1.1" 204 0
2025-12-05 20:02:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/879/4c9260dff41b43ecbd2162eb1dfa6124/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/773/472e1714abac402bae8007ee0df0de76/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:31 [ayvens.es] INFO: Found listing with ID: 1788874
2025-12-05 20:02:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788874%27 HTTP/1.1" 200 None
2025-12-05 20:02:31 [ayvens.es] INFO: Scrape type for 1788874: 1
2025-12-05 20:02:31 [ayvens.es] INFO: Extracting data for car ID: 1788874
2025-12-05 20:02:31 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 002ea523-418a-587f-8b11-9220b3f39387 with identifier: 1788874
2025-12-05 20:02:31 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 002ea523-418a-587f-8b11-9220b3f39387 with scrape type 1
2025-12-05 20:02:31 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 002ea523-418a-587f-8b11-9220b3f39387
2025-12-05 20:02:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F621%252F1e0515d4d90c466cad2fb7fb2271d0c8%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/621/1e0515d4d90c466cad2fb7fb2271d0c8/images/large/1.jpg not downloaded yet
2025-12-05 20:02:31 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:31 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:31 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:31 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b84050>
2025-12-05 20:02:31 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:31 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:31 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:02:31 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:31 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:31 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:32 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_9bdb369d-b46e-5f62-9ae1-f9acacab1ec0/9bdb369d-b46e-5f62-9ae1-f9acacab1ec0_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F820%252F49a4eb301b914ec697039e1ca43083f5%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/820/49a4eb301b914ec697039e1ca43083f5/images/large/1.jpg saved to cache
2025-12-05 20:02:32 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 9bdb369d-b46e-5f62-9ae1-f9acacab1ec0
2025-12-05 20:02:32 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 9bdb369d-b46e-5f62-9ae1-f9acacab1ec0, skipping ID generation
2025-12-05 20:02:32 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9bdb369d-b46e-5f62-9ae1-f9acacab1ec0 with scrape type 1
2025-12-05 20:02:32 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 9bdb369d-b46e-5f62-9ae1-f9acacab1ec0 from tracking dictionaries
2025-12-05 20:02:32 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 9bdb369d-b46e-5f62-9ae1-f9acacab1ec0 sending to next pipeline
2025-12-05 20:02:32 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 9bdb369d-b46e-5f62-9ae1-f9acacab1ec0, identifier: 1788865
2025-12-05 20:02:32 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:32 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:32 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:32 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0d47150>
2025-12-05 20:02:32 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:32 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:32 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:02:32 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:32 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:32 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:32 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_74ac6544-42e5-5bab-995a-813dd5704d95/74ac6544-42e5-5bab-995a-813dd5704d95_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F725%252F2bb95cb1d8fc4d7c8e58d9991cee745f%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/725/2bb95cb1d8fc4d7c8e58d9991cee745f/images/large/1.jpg saved to cache
2025-12-05 20:02:32 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 74ac6544-42e5-5bab-995a-813dd5704d95
2025-12-05 20:02:32 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 74ac6544-42e5-5bab-995a-813dd5704d95, skipping ID generation
2025-12-05 20:02:32 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 74ac6544-42e5-5bab-995a-813dd5704d95 with scrape type 1
2025-12-05 20:02:32 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 74ac6544-42e5-5bab-995a-813dd5704d95 from tracking dictionaries
2025-12-05 20:02:32 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 74ac6544-42e5-5bab-995a-813dd5704d95 sending to next pipeline
2025-12-05 20:02:32 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 74ac6544-42e5-5bab-995a-813dd5704d95, identifier: 1788866
2025-12-05 20:02:32 [ayvens.es] INFO: Found listing with ID: 1788875
2025-12-05 20:02:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788875%27 HTTP/1.1" 200 None
2025-12-05 20:02:32 [ayvens.es] INFO: Scrape type for 1788875: 1
2025-12-05 20:02:32 [ayvens.es] INFO: Extracting data for car ID: 1788875
2025-12-05 20:02:32 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7 with identifier: 1788875
2025-12-05 20:02:32 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7 with scrape type 1
2025-12-05 20:02:32 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7
2025-12-05 20:02:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F335%252F88ef8b22c6ba46ce8b95d9a47a64cbc1%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/335/88ef8b22c6ba46ce8b95d9a47a64cbc1/images/large/1.jpg not downloaded yet
2025-12-05 20:02:32 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 9bdb369d-b46e-5f62-9ae1-f9acacab1ec0 with identifier: 1788865
2025-12-05 20:02:32 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 9bdb369d-b46e-5f62-9ae1-f9acacab1ec0 with identifier: 1788865
2025-12-05 20:02:32 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 9bdb369d-b46e-5f62-9ae1-f9acacab1ec0 with identifier: 1788865 to the API
2025-12-05 20:02:32 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:32 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:33 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/820/49a4eb301b914ec697039e1ca43083f5/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 129,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '9bdb369d-b46e-5f62-9ae1-f9acacab1ec0',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SEAT',
 'model': 'ATECA',
 'odometer': 78865,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_9bdb369d-b46e-5f62-9ae1-f9acacab1ec0/9bdb369d-b46e-5f62-9ae1-f9acacab1ec0_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 337092,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/820/49a4eb301b914ec697039e1ca43083f5/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-06-24',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SEAT ATECA 2.0 TDI 85kW (115CV) S&S Style Go',
 'transmission': 'manual',
 'trim': '2.0 TDI 85kW (115CV) S&S Style Go',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788865%27 HTTP/1.1" 200 None
2025-12-05 20:02:33 [ayvens.es] INFO: Saving data for 1788865: {'created_time': 1764964953.031379, 'last_price_update_time': 1764964953.031388, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788865') HTTP/1.1" 204 0
2025-12-05 20:02:33 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 74ac6544-42e5-5bab-995a-813dd5704d95 with identifier: 1788866
2025-12-05 20:02:33 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 74ac6544-42e5-5bab-995a-813dd5704d95 with identifier: 1788866
2025-12-05 20:02:33 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 74ac6544-42e5-5bab-995a-813dd5704d95 with identifier: 1788866 to the API
2025-12-05 20:02:33 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:33 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:33 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/725/2bb95cb1d8fc4d7c8e58d9991cee745f/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 116,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '74ac6544-42e5-5bab-995a-813dd5704d95',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': '2008',
 'odometer': 39501,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_74ac6544-42e5-5bab-995a-813dd5704d95/74ac6544-42e5-5bab-995a-813dd5704d95_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 479572,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/725/2bb95cb1d8fc4d7c8e58d9991cee745f/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-11-23',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT 2008 Active Pack BlueHDI 81kW (110CV)',
 'transmission': 'manual',
 'trim': 'Active Pack BlueHDI 81kW (110CV)',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788866%27 HTTP/1.1" 200 None
2025-12-05 20:02:33 [ayvens.es] INFO: Saving data for 1788866: {'created_time': 1764964953.361902, 'last_price_update_time': 1764964953.361911, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788866') HTTP/1.1" 204 0
2025-12-05 20:02:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/685/2580bca87c994d1a9b7e69571dcecd7b/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/690/04126256354a4c998930421bbfbfab0f/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:33 [ayvens.es] INFO: Found listing with ID: 1788876
2025-12-05 20:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788876%27 HTTP/1.1" 200 None
2025-12-05 20:02:33 [ayvens.es] INFO: Scrape type for 1788876: 1
2025-12-05 20:02:33 [ayvens.es] INFO: Extracting data for car ID: 1788876
2025-12-05 20:02:33 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: dc759675-267d-5217-91f1-b67b3e68f9e4 with identifier: 1788876
2025-12-05 20:02:33 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item dc759675-267d-5217-91f1-b67b3e68f9e4 with scrape type 1
2025-12-05 20:02:33 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item dc759675-267d-5217-91f1-b67b3e68f9e4
2025-12-05 20:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F251%252F6eebcc6e9ad141c288c23936c9dc67b7%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/251/6eebcc6e9ad141c288c23936c9dc67b7/images/large/1.jpg not downloaded yet
2025-12-05 20:02:33 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:33 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:33 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:33 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a60590>
2025-12-05 20:02:33 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:33 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:33 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:02:33 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:33 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:33 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:33 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_e3226609-95c1-50fe-a295-d8ca46440a6f/e3226609-95c1-50fe-a295-d8ca46440a6f_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F879%252F4c9260dff41b43ecbd2162eb1dfa6124%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/879/4c9260dff41b43ecbd2162eb1dfa6124/images/large/1.jpg saved to cache
2025-12-05 20:02:33 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item e3226609-95c1-50fe-a295-d8ca46440a6f
2025-12-05 20:02:33 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: e3226609-95c1-50fe-a295-d8ca46440a6f, skipping ID generation
2025-12-05 20:02:33 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e3226609-95c1-50fe-a295-d8ca46440a6f with scrape type 1
2025-12-05 20:02:33 [crawlers.middlewares.photo_download_middleware] INFO: Removing item e3226609-95c1-50fe-a295-d8ca46440a6f from tracking dictionaries
2025-12-05 20:02:33 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item e3226609-95c1-50fe-a295-d8ca46440a6f sending to next pipeline
2025-12-05 20:02:33 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: e3226609-95c1-50fe-a295-d8ca46440a6f, identifier: 1788868
2025-12-05 20:02:33 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:33 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:33 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:33 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0d9bb10>
2025-12-05 20:02:33 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:33 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:34 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:02:34 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:34 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:34 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:34 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_d8e6f5f6-fa7e-5c6d-91a2-351921d13f93/d8e6f5f6-fa7e-5c6d-91a2-351921d13f93_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F773%252F472e1714abac402bae8007ee0df0de76%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/773/472e1714abac402bae8007ee0df0de76/images/large/1.jpg saved to cache
2025-12-05 20:02:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item d8e6f5f6-fa7e-5c6d-91a2-351921d13f93
2025-12-05 20:02:34 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: d8e6f5f6-fa7e-5c6d-91a2-351921d13f93, skipping ID generation
2025-12-05 20:02:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item d8e6f5f6-fa7e-5c6d-91a2-351921d13f93 with scrape type 1
2025-12-05 20:02:34 [crawlers.middlewares.photo_download_middleware] INFO: Removing item d8e6f5f6-fa7e-5c6d-91a2-351921d13f93 from tracking dictionaries
2025-12-05 20:02:34 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item d8e6f5f6-fa7e-5c6d-91a2-351921d13f93 sending to next pipeline
2025-12-05 20:02:34 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: d8e6f5f6-fa7e-5c6d-91a2-351921d13f93, identifier: 1788867
2025-12-05 20:02:34 [ayvens.es] INFO: Found listing with ID: 1788877
2025-12-05 20:02:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788877%27 HTTP/1.1" 200 None
2025-12-05 20:02:34 [ayvens.es] INFO: Scrape type for 1788877: 1
2025-12-05 20:02:34 [ayvens.es] INFO: Extracting data for car ID: 1788877
2025-12-05 20:02:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 95d3b077-c1b2-599b-82ef-98a3d47282dc with identifier: 1788877
2025-12-05 20:02:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 95d3b077-c1b2-599b-82ef-98a3d47282dc with scrape type 1
2025-12-05 20:02:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 95d3b077-c1b2-599b-82ef-98a3d47282dc
2025-12-05 20:02:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F582%252F8c9f6bc52ca74f0a909da83ce2b9d2f4%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/582/8c9f6bc52ca74f0a909da83ce2b9d2f4/images/large/1.jpg not downloaded yet
2025-12-05 20:02:34 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: e3226609-95c1-50fe-a295-d8ca46440a6f with identifier: 1788868
2025-12-05 20:02:34 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: e3226609-95c1-50fe-a295-d8ca46440a6f with identifier: 1788868
2025-12-05 20:02:34 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: e3226609-95c1-50fe-a295-d8ca46440a6f with identifier: 1788868 to the API
2025-12-05 20:02:34 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:34 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:34 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/879/4c9260dff41b43ecbd2162eb1dfa6124/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 120,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': 'e3226609-95c1-50fe-a295-d8ca46440a6f',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SEAT',
 'model': 'ARONA',
 'odometer': 45912,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_e3226609-95c1-50fe-a295-d8ca46440a6f/e3226609-95c1-50fe-a295-d8ca46440a6f_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 348205,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/879/4c9260dff41b43ecbd2162eb1dfa6124/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-07-30',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SEAT ARONA 1.0 TSI 81kW (110CV) Style Go2',
 'transmission': 'manual',
 'trim': '1.0 TSI 81kW (110CV) Style Go2',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788868%27 HTTP/1.1" 200 None
2025-12-05 20:02:34 [ayvens.es] INFO: Saving data for 1788868: {'created_time': 1764964954.687889, 'last_price_update_time': 1764964954.687901, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788868') HTTP/1.1" 204 0
2025-12-05 20:02:34 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: d8e6f5f6-fa7e-5c6d-91a2-351921d13f93 with identifier: 1788867
2025-12-05 20:02:34 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: d8e6f5f6-fa7e-5c6d-91a2-351921d13f93 with identifier: 1788867
2025-12-05 20:02:34 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: d8e6f5f6-fa7e-5c6d-91a2-351921d13f93 with identifier: 1788867 to the API
2025-12-05 20:02:34 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:34 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:35 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/773/472e1714abac402bae8007ee0df0de76/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 122,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': 'd8e6f5f6-fa7e-5c6d-91a2-351921d13f93',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'NISSAN',
 'model': 'MICRA',
 'odometer': 41899,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_d8e6f5f6-fa7e-5c6d-91a2-351921d13f93/d8e6f5f6-fa7e-5c6d-91a2-351921d13f93_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 339090,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/773/472e1714abac402bae8007ee0df0de76/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-08-16',
 'seats_number': 5,
 'seller_name': None,
 'title': 'NISSAN MICRA IG-T 68 kW (92 CV) E6D-F N-Design Black',
 'transmission': 'manual',
 'trim': 'IG-T 68 kW (92 CV) E6D-F N-Design Black',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788867%27 HTTP/1.1" 200 None
2025-12-05 20:02:35 [ayvens.es] INFO: Saving data for 1788867: {'created_time': 1764964955.057017, 'last_price_update_time': 1764964955.057026, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788867') HTTP/1.1" 204 0
2025-12-05 20:02:35 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/722/5e016041c81c460b831ab26221f49071/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:35 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/352/65241b5880784f65af6b7d97e3350640/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:35 [ayvens.es] INFO: Found listing with ID: 1788878
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788878%27 HTTP/1.1" 200 None
2025-12-05 20:02:35 [ayvens.es] INFO: Scrape type for 1788878: 1
2025-12-05 20:02:35 [ayvens.es] INFO: Extracting data for car ID: 1788878
2025-12-05 20:02:35 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 75df8548-1322-588d-8461-d03ee84a3494 with identifier: 1788878
2025-12-05 20:02:35 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 75df8548-1322-588d-8461-d03ee84a3494 with scrape type 1
2025-12-05 20:02:35 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 75df8548-1322-588d-8461-d03ee84a3494
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F638%252Fe460d359bdce4c87983d063c43b7341a%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/638/e460d359bdce4c87983d063c43b7341a/images/large/1.jpg not downloaded yet
2025-12-05 20:02:35 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:35 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:35 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:35 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0548b50>
2025-12-05 20:02:35 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1971
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:35 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_66348a8e-f41f-5373-a724-6b5b4f0bfa77/66348a8e-f41f-5373-a724-6b5b4f0bfa77_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F685%252F2580bca87c994d1a9b7e69571dcecd7b%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/685/2580bca87c994d1a9b7e69571dcecd7b/images/large/1.jpg saved to cache
2025-12-05 20:02:35 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 66348a8e-f41f-5373-a724-6b5b4f0bfa77
2025-12-05 20:02:35 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 66348a8e-f41f-5373-a724-6b5b4f0bfa77, skipping ID generation
2025-12-05 20:02:35 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 66348a8e-f41f-5373-a724-6b5b4f0bfa77 with scrape type 1
2025-12-05 20:02:35 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 66348a8e-f41f-5373-a724-6b5b4f0bfa77 from tracking dictionaries
2025-12-05 20:02:35 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 66348a8e-f41f-5373-a724-6b5b4f0bfa77 sending to next pipeline
2025-12-05 20:02:35 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 66348a8e-f41f-5373-a724-6b5b4f0bfa77, identifier: 1788869
2025-12-05 20:02:35 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:35 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:35 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:35 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b04842d0>
2025-12-05 20:02:35 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:35 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_3ab59d01-2d64-5645-831e-54378e0c3941/3ab59d01-2d64-5645-831e-54378e0c3941_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F690%252F04126256354a4c998930421bbfbfab0f%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/690/04126256354a4c998930421bbfbfab0f/images/large/1.jpg saved to cache
2025-12-05 20:02:35 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 3ab59d01-2d64-5645-831e-54378e0c3941
2025-12-05 20:02:35 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 3ab59d01-2d64-5645-831e-54378e0c3941, skipping ID generation
2025-12-05 20:02:35 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 3ab59d01-2d64-5645-831e-54378e0c3941 with scrape type 1
2025-12-05 20:02:35 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 3ab59d01-2d64-5645-831e-54378e0c3941 from tracking dictionaries
2025-12-05 20:02:35 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 3ab59d01-2d64-5645-831e-54378e0c3941 sending to next pipeline
2025-12-05 20:02:35 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 3ab59d01-2d64-5645-831e-54378e0c3941, identifier: 1788870
2025-12-05 20:02:35 [ayvens.es] INFO: Found listing with ID: 1788879
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788879%27 HTTP/1.1" 200 None
2025-12-05 20:02:35 [ayvens.es] INFO: Scrape type for 1788879: 1
2025-12-05 20:02:36 [ayvens.es] INFO: Extracting data for car ID: 1788879
2025-12-05 20:02:36 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: aebb980e-1715-5138-a761-3e49ade1d6a1 with identifier: 1788879
2025-12-05 20:02:36 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item aebb980e-1715-5138-a761-3e49ade1d6a1 with scrape type 1
2025-12-05 20:02:36 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item aebb980e-1715-5138-a761-3e49ade1d6a1
2025-12-05 20:02:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F383%252F17aa565583e44433ae3a5d195ec7d239%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:36 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/383/17aa565583e44433ae3a5d195ec7d239/images/large/1.jpg not downloaded yet
2025-12-05 20:02:36 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 66348a8e-f41f-5373-a724-6b5b4f0bfa77 with identifier: 1788869
2025-12-05 20:02:36 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 66348a8e-f41f-5373-a724-6b5b4f0bfa77 with identifier: 1788869
2025-12-05 20:02:36 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 66348a8e-f41f-5373-a724-6b5b4f0bfa77 with identifier: 1788869 to the API
2025-12-05 20:02:36 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:36 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:36 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/685/2580bca87c994d1a9b7e69571dcecd7b/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 104,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '66348a8e-f41f-5373-a724-6b5b4f0bfa77',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'POLO',
 'odometer': 85582,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_66348a8e-f41f-5373-a724-6b5b4f0bfa77/66348a8e-f41f-5373-a724-6b5b4f0bfa77_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 324221,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/685/2580bca87c994d1a9b7e69571dcecd7b/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-04-29',
 'seats_number': 5,
 'seller_name': None,
 'title': 'VOLKSWAGEN POLO Advance 1.0 TSI 70kW (95CV)',
 'transmission': 'manual',
 'trim': 'Advance 1.0 TSI 70kW (95CV)',
 'vin': None,
 'year': 2019}
2025-12-05 20:02:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788869%27 HTTP/1.1" 200 None
2025-12-05 20:02:36 [ayvens.es] INFO: Saving data for 1788869: {'created_time': 1764964956.323987, 'last_price_update_time': 1764964956.323997, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788869') HTTP/1.1" 204 0
2025-12-05 20:02:36 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 3ab59d01-2d64-5645-831e-54378e0c3941 with identifier: 1788870
2025-12-05 20:02:36 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 3ab59d01-2d64-5645-831e-54378e0c3941 with identifier: 1788870
2025-12-05 20:02:36 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 3ab59d01-2d64-5645-831e-54378e0c3941 with identifier: 1788870 to the API
2025-12-05 20:02:36 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:36 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:36 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/690/04126256354a4c998930421bbfbfab0f/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 117,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.2L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '3ab59d01-2d64-5645-831e-54378e0c3941',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'OPEL',
 'model': 'CORSA',
 'odometer': 20150,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_3ab59d01-2d64-5645-831e-54378e0c3941/3ab59d01-2d64-5645-831e-54378e0c3941_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 355534,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/690/04126256354a4c998930421bbfbfab0f/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2022-08-05',
 'seats_number': 5,
 'seller_name': None,
 'title': 'OPEL CORSA 1.2T XHL 74kW (100CV) Edition',
 'transmission': 'manual',
 'trim': '1.2T XHL 74kW (100CV) Edition',
 'vin': None,
 'year': 2022}
2025-12-05 20:02:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788870%27 HTTP/1.1" 200 None
2025-12-05 20:02:36 [ayvens.es] INFO: Saving data for 1788870: {'created_time': 1764964956.792018, 'last_price_update_time': 1764964956.79203, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788870') HTTP/1.1" 204 0
2025-12-05 20:02:36 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/621/1e0515d4d90c466cad2fb7fb2271d0c8/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:36 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/551/fba99fad1f474c5496eb3ff911e59b95/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:36 [ayvens.es] INFO: Found listing with ID: 1788880
2025-12-05 20:02:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788880%27 HTTP/1.1" 200 None
2025-12-05 20:02:36 [ayvens.es] INFO: Scrape type for 1788880: 1
2025-12-05 20:02:36 [ayvens.es] INFO: Extracting data for car ID: 1788880
2025-12-05 20:02:36 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: eab09ec2-d9f2-5748-9650-a123712d465a with identifier: 1788880
2025-12-05 20:02:36 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item eab09ec2-d9f2-5748-9650-a123712d465a with scrape type 1
2025-12-05 20:02:36 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item eab09ec2-d9f2-5748-9650-a123712d465a
2025-12-05 20:02:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F445%252Fef6390a1414040699505a24e65cb34b6%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:36 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/445/ef6390a1414040699505a24e65cb34b6/images/large/1.jpg not downloaded yet
2025-12-05 20:02:36 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:36 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:36 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:36 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b053bb10>
2025-12-05 20:02:36 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:36 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:37 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:02:37 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:37 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:37 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:37 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_be1fab5b-2d09-510b-b699-499311efdfab/be1fab5b-2d09-510b-b699-499311efdfab_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F722%252F5e016041c81c460b831ab26221f49071%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:37 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/722/5e016041c81c460b831ab26221f49071/images/large/1.jpg saved to cache
2025-12-05 20:02:37 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item be1fab5b-2d09-510b-b699-499311efdfab
2025-12-05 20:02:37 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: be1fab5b-2d09-510b-b699-499311efdfab, skipping ID generation
2025-12-05 20:02:37 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item be1fab5b-2d09-510b-b699-499311efdfab with scrape type 1
2025-12-05 20:02:37 [crawlers.middlewares.photo_download_middleware] INFO: Removing item be1fab5b-2d09-510b-b699-499311efdfab from tracking dictionaries
2025-12-05 20:02:37 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item be1fab5b-2d09-510b-b699-499311efdfab sending to next pipeline
2025-12-05 20:02:37 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: be1fab5b-2d09-510b-b699-499311efdfab, identifier: 1788872
2025-12-05 20:02:37 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:37 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:37 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:37 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b06c6ed0>
2025-12-05 20:02:37 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:37 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:37 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:02:37 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:37 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:37 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:37 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_fdc12952-17a4-5be1-a5d9-c7988ff18dbe/fdc12952-17a4-5be1-a5d9-c7988ff18dbe_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F352%252F65241b5880784f65af6b7d97e3350640%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:37 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/352/65241b5880784f65af6b7d97e3350640/images/large/1.jpg saved to cache
2025-12-05 20:02:37 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item fdc12952-17a4-5be1-a5d9-c7988ff18dbe
2025-12-05 20:02:37 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: fdc12952-17a4-5be1-a5d9-c7988ff18dbe, skipping ID generation
2025-12-05 20:02:37 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item fdc12952-17a4-5be1-a5d9-c7988ff18dbe with scrape type 1
2025-12-05 20:02:37 [crawlers.middlewares.photo_download_middleware] INFO: Removing item fdc12952-17a4-5be1-a5d9-c7988ff18dbe from tracking dictionaries
2025-12-05 20:02:37 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item fdc12952-17a4-5be1-a5d9-c7988ff18dbe sending to next pipeline
2025-12-05 20:02:37 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: fdc12952-17a4-5be1-a5d9-c7988ff18dbe, identifier: 1788871
2025-12-05 20:02:37 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/335/88ef8b22c6ba46ce8b95d9a47a64cbc1/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:37 [ayvens.es] INFO: Found listing with ID: 1788881
2025-12-05 20:02:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788881%27 HTTP/1.1" 200 None
2025-12-05 20:02:37 [ayvens.es] INFO: Scrape type for 1788881: 1
2025-12-05 20:02:37 [ayvens.es] INFO: Extracting data for car ID: 1788881
2025-12-05 20:02:37 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: f7186b82-b603-5803-8dc8-4be83117e4d6 with identifier: 1788881
2025-12-05 20:02:37 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f7186b82-b603-5803-8dc8-4be83117e4d6 with scrape type 1
2025-12-05 20:02:37 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item f7186b82-b603-5803-8dc8-4be83117e4d6
2025-12-05 20:02:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F427%252Ffda6148e6a8746a59d9b7b3b9a20fbda%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:37 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/427/fda6148e6a8746a59d9b7b3b9a20fbda/images/large/1.jpg not downloaded yet
2025-12-05 20:02:37 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: be1fab5b-2d09-510b-b699-499311efdfab with identifier: 1788872
2025-12-05 20:02:37 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: be1fab5b-2d09-510b-b699-499311efdfab with identifier: 1788872
2025-12-05 20:02:37 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'renault' containing 'renault' and model 'kangoo express' containing 'express'
2025-12-05 20:02:37 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: be1fab5b-2d09-510b-b699-499311efdfab spider: ayvens.es
2025-12-05 20:02:37 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: be1fab5b-2d09-510b-b699-499311efdfab due to rule: makes_models_not_allowed.json, error: Rejected due to make 'renault' containing 'renault' and model 'kangoo express' containing 'express'
{'auction_title': '',
 'c_o_2_emission_value': 117,
 'color': 'Pintura',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'be1fab5b-2d09-510b-b699-499311efdfab',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'KANGOO EXPRESS',
 'odometer': 43133,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_be1fab5b-2d09-510b-b699-499311efdfab/be1fab5b-2d09-510b-b699-499311efdfab_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 348850,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/722/5e016041c81c460b831ab26221f49071/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-08-13',
 'seats_number': 2,
 'seller_name': None,
 'title': 'RENAULT KANGOO EXPRESS Maxi 2p Blue dCi 70 kW (95CV)',
 'transmission': 'manual',
 'trim': 'Maxi 2p Blue dCi 70 kW (95CV)',
 'vin': None,
 'year': 2020}
2025-12-05 20:02:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788872%27 HTTP/1.1" 200 None
2025-12-05 20:02:38 [ayvens.es] INFO: Saving data for 1788872: {'created_time': 1764964958.016317, 'last_price_update_time': 1764964958.016329, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788872') HTTP/1.1" 204 0
2025-12-05 20:02:38 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: fdc12952-17a4-5be1-a5d9-c7988ff18dbe with identifier: 1788871
2025-12-05 20:02:38 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: fdc12952-17a4-5be1-a5d9-c7988ff18dbe with identifier: 1788871
2025-12-05 20:02:38 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: fdc12952-17a4-5be1-a5d9-c7988ff18dbe with identifier: 1788871 to the API
2025-12-05 20:02:38 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:38 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:38 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/352/65241b5880784f65af6b7d97e3350640/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 93,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'fdc12952-17a4-5be1-a5d9-c7988ff18dbe',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': '2008',
 'odometer': 94725,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_fdc12952-17a4-5be1-a5d9-c7988ff18dbe/fdc12952-17a4-5be1-a5d9-c7988ff18dbe_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 343495,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/352/65241b5880784f65af6b7d97e3350640/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-08-12',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT 2008 Signature BlueHDi 73KW (100CV)',
 'transmission': 'manual',
 'trim': 'Signature BlueHDi 73KW (100CV)',
 'vin': None,
 'year': 2019}
2025-12-05 20:02:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788871%27 HTTP/1.1" 200 None
2025-12-05 20:02:38 [ayvens.es] INFO: Saving data for 1788871: {'created_time': 1764964958.441065, 'last_price_update_time': 1764964958.441074, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788871') HTTP/1.1" 204 0
2025-12-05 20:02:38 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/251/6eebcc6e9ad141c288c23936c9dc67b7/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:38 [ayvens.es] INFO: Found listing with ID: 1788882
2025-12-05 20:02:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788882%27 HTTP/1.1" 200 None
2025-12-05 20:02:38 [ayvens.es] INFO: Scrape type for 1788882: 1
2025-12-05 20:02:38 [ayvens.es] INFO: Extracting data for car ID: 1788882
2025-12-05 20:02:38 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: ec7f1733-6376-594d-8eaa-74bf1b70d714 with identifier: 1788882
2025-12-05 20:02:38 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ec7f1733-6376-594d-8eaa-74bf1b70d714 with scrape type 1
2025-12-05 20:02:38 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item ec7f1733-6376-594d-8eaa-74bf1b70d714
2025-12-05 20:02:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F845%252F454fb9c25db84227bb23de473454048c%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:38 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/845/454fb9c25db84227bb23de473454048c/images/large/1.jpg not downloaded yet
2025-12-05 20:02:38 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:38 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:38 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:38 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b44a10>
2025-12-05 20:02:38 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:38 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:38 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1971
2025-12-05 20:02:38 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:38 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:38 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:38 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_002ea523-418a-587f-8b11-9220b3f39387/002ea523-418a-587f-8b11-9220b3f39387_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F621%252F1e0515d4d90c466cad2fb7fb2271d0c8%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:38 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/621/1e0515d4d90c466cad2fb7fb2271d0c8/images/large/1.jpg saved to cache
2025-12-05 20:02:38 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 002ea523-418a-587f-8b11-9220b3f39387
2025-12-05 20:02:38 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 002ea523-418a-587f-8b11-9220b3f39387, skipping ID generation
2025-12-05 20:02:38 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 002ea523-418a-587f-8b11-9220b3f39387 with scrape type 1
2025-12-05 20:02:38 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 002ea523-418a-587f-8b11-9220b3f39387 from tracking dictionaries
2025-12-05 20:02:38 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 002ea523-418a-587f-8b11-9220b3f39387 sending to next pipeline
2025-12-05 20:02:38 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 002ea523-418a-587f-8b11-9220b3f39387, identifier: 1788874
2025-12-05 20:02:38 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:38 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:38 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:38 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b45f10>
2025-12-05 20:02:38 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:38 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:39 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:02:39 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:39 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:39 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:39 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_6ac01f5c-c3dd-54c9-927c-7b8f74db3393/6ac01f5c-c3dd-54c9-927c-7b8f74db3393_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F551%252Ffba99fad1f474c5496eb3ff911e59b95%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:39 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/551/fba99fad1f474c5496eb3ff911e59b95/images/large/1.jpg saved to cache
2025-12-05 20:02:39 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 6ac01f5c-c3dd-54c9-927c-7b8f74db3393
2025-12-05 20:02:39 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 6ac01f5c-c3dd-54c9-927c-7b8f74db3393, skipping ID generation
2025-12-05 20:02:39 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6ac01f5c-c3dd-54c9-927c-7b8f74db3393 with scrape type 1
2025-12-05 20:02:39 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 6ac01f5c-c3dd-54c9-927c-7b8f74db3393 from tracking dictionaries
2025-12-05 20:02:39 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 6ac01f5c-c3dd-54c9-927c-7b8f74db3393 sending to next pipeline
2025-12-05 20:02:39 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 6ac01f5c-c3dd-54c9-927c-7b8f74db3393, identifier: 1788873
2025-12-05 20:02:39 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 002ea523-418a-587f-8b11-9220b3f39387 with identifier: 1788874
2025-12-05 20:02:39 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 002ea523-418a-587f-8b11-9220b3f39387 with identifier: 1788874
2025-12-05 20:02:39 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 002ea523-418a-587f-8b11-9220b3f39387 with identifier: 1788874 to the API
2025-12-05 20:02:39 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:39 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:39 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/621/1e0515d4d90c466cad2fb7fb2271d0c8/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 122,
 'color': 'Plata',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '002ea523-418a-587f-8b11-9220b3f39387',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'PASSAT',
 'odometer': 110818,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_002ea523-418a-587f-8b11-9220b3f39387/002ea523-418a-587f-8b11-9220b3f39387_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 330525,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/621/1e0515d4d90c466cad2fb7fb2271d0c8/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-11-10',
 'seats_number': 5,
 'seller_name': None,
 'title': 'VOLKSWAGEN PASSAT Business 2.0 TDI 110kW (150CV)',
 'transmission': 'manual',
 'trim': 'Business 2.0 TDI 110kW (150CV)',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788874%27 HTTP/1.1" 200 None
2025-12-05 20:02:39 [ayvens.es] INFO: Saving data for 1788874: {'created_time': 1764964959.678372, 'last_price_update_time': 1764964959.678383, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788874') HTTP/1.1" 204 0
2025-12-05 20:02:39 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 6ac01f5c-c3dd-54c9-927c-7b8f74db3393 with identifier: 1788873
2025-12-05 20:02:39 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 6ac01f5c-c3dd-54c9-927c-7b8f74db3393 with identifier: 1788873
2025-12-05 20:02:39 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'renault' containing 'renault' and model 'kangoo express' containing 'express'
2025-12-05 20:02:39 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 6ac01f5c-c3dd-54c9-927c-7b8f74db3393 spider: ayvens.es
2025-12-05 20:02:40 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 6ac01f5c-c3dd-54c9-927c-7b8f74db3393 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'renault' containing 'renault' and model 'kangoo express' containing 'express'
{'auction_title': '',
 'c_o_2_emission_value': 117,
 'color': 'Pintura',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '6ac01f5c-c3dd-54c9-927c-7b8f74db3393',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'KANGOO EXPRESS',
 'odometer': 59465,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_6ac01f5c-c3dd-54c9-927c-7b8f74db3393/6ac01f5c-c3dd-54c9-927c-7b8f74db3393_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 341569,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/551/fba99fad1f474c5496eb3ff911e59b95/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-07-01',
 'seats_number': 2,
 'seller_name': None,
 'title': 'RENAULT KANGOO EXPRESS Maxi 2p Blue dCi 70 kW (95CV)',
 'transmission': 'manual',
 'trim': 'Maxi 2p Blue dCi 70 kW (95CV)',
 'vin': None,
 'year': 2020}
2025-12-05 20:02:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788873%27 HTTP/1.1" 200 None
2025-12-05 20:02:40 [ayvens.es] INFO: Saving data for 1788873: {'created_time': 1764964960.048128, 'last_price_update_time': 1764964960.048137, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788873') HTTP/1.1" 204 0
2025-12-05 20:02:40 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:40 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:40 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:40 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b04b7b90>
2025-12-05 20:02:40 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:40 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:40 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1971
2025-12-05 20:02:40 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:40 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:40 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:40 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7/fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F335%252F88ef8b22c6ba46ce8b95d9a47a64cbc1%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:40 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/335/88ef8b22c6ba46ce8b95d9a47a64cbc1/images/large/1.jpg saved to cache
2025-12-05 20:02:40 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7
2025-12-05 20:02:40 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7, skipping ID generation
2025-12-05 20:02:40 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7 with scrape type 1
2025-12-05 20:02:40 [crawlers.middlewares.photo_download_middleware] INFO: Removing item fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7 from tracking dictionaries
2025-12-05 20:02:40 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7 sending to next pipeline
2025-12-05 20:02:40 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7, identifier: 1788875
2025-12-05 20:02:40 [ayvens.es] INFO: Found listing with ID: 1788883
2025-12-05 20:02:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788883%27 HTTP/1.1" 200 None
2025-12-05 20:02:40 [ayvens.es] INFO: Scrape type for 1788883: 1
2025-12-05 20:02:40 [ayvens.es] INFO: Extracting data for car ID: 1788883
2025-12-05 20:02:40 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: caa8d3ba-c53d-5300-922d-0ad7a28c1449 with identifier: 1788883
2025-12-05 20:02:40 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item caa8d3ba-c53d-5300-922d-0ad7a28c1449 with scrape type 1
2025-12-05 20:02:40 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item caa8d3ba-c53d-5300-922d-0ad7a28c1449
2025-12-05 20:02:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F105%252F8d338a1beec749a0893038782695b337%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:40 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/105/8d338a1beec749a0893038782695b337/images/large/1.jpg not downloaded yet
2025-12-05 20:02:40 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7 with identifier: 1788875
2025-12-05 20:02:40 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7 with identifier: 1788875
2025-12-05 20:02:40 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'banquise', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'citroën', 'model': 'jumpy', 'trim': 'talla xl bluehdi 120 s&s 6v control', 'title': 'citroën jumpy talla xl bluehdi 120 s&s 6v control', 'price': 0, 'odometer': 230670, 'odometer_reading_unit': 'km', 'registration_date': '2021-10-11', 'year': 2021, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 4, 'seats_number': 2, 'vin': None, 'license_plate': None, 'engine': '2.0l', 'c_o_2_emission_value': 190, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7/fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/335/88ef8b22c6ba46ce8b95d9a47a64cbc1/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 320748}], 'current_year_diff': 4, 'age_in_months': 49, 'make_model': 'citroën jumpy', 'sanitized_make': 'citron', 'odometer_in_km': 230670}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 49,
     'auction_title': '',
     'c_o_2_emission_value': 190,
     'color': 'banquise',
     'currency': 'eur',
     'current_year_diff': 4,
     'doors_number': 4,
     'engine': '2.0l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': 'fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'citroën',
     'make_model': 'citroën jumpy',
     'model': 'jumpy',
     'odometer': 230670,
     'odometer_in_km': 230670,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7/fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 320748,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/335/88ef8b22c6ba46ce8b95d9a47a64cbc1/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2021-10-11',
     'sanitized_make': 'citron',
     'seats_number': 2,
     'seller_name': None,
     'title': 'citroën jumpy talla xl bluehdi 120 s&s 6v control',
     'transmission': 'manual',
     'trim': 'talla xl bluehdi 120 s&s 6v control',
     'vin': None,
     'year': 2021}
2025-12-05 20:02:40 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE failed for item: fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7 spider: ayvens.es
2025-12-05 20:02:40 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7 due to rule: high_mileage.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'banquise', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'citroën', 'model': 'jumpy', 'trim': 'talla xl bluehdi 120 s&s 6v control', 'title': 'citroën jumpy talla xl bluehdi 120 s&s 6v control', 'price': 0, 'odometer': 230670, 'odometer_reading_unit': 'km', 'registration_date': '2021-10-11', 'year': 2021, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 4, 'seats_number': 2, 'vin': None, 'license_plate': None, 'engine': '2.0l', 'c_o_2_emission_value': 190, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7/fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/335/88ef8b22c6ba46ce8b95d9a47a64cbc1/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 320748}], 'current_year_diff': 4, 'age_in_months': 49, 'make_model': 'citroën jumpy', 'sanitized_make': 'citron', 'odometer_in_km': 230670}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 49,
     'auction_title': '',
     'c_o_2_emission_value': 190,
     'color': 'banquise',
     'currency': 'eur',
     'current_year_diff': 4,
     'doors_number': 4,
     'engine': '2.0l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': 'fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'citroën',
     'make_model': 'citroën jumpy',
     'model': 'jumpy',
     'odometer': 230670,
     'odometer_in_km': 230670,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7/fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 320748,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/335/88ef8b22c6ba46ce8b95d9a47a64cbc1/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2021-10-11',
     'sanitized_make': 'citron',
     'seats_number': 2,
     'seller_name': None,
     'title': 'citroën jumpy talla xl bluehdi 120 s&s 6v control',
     'transmission': 'manual',
     'trim': 'talla xl bluehdi 120 s&s 6v control',
     'vin': None,
     'year': 2021}
{'auction_title': '',
 'c_o_2_emission_value': 190,
 'color': 'Banquise',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'CITROËN',
 'model': 'JUMPY',
 'odometer': 230670,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7/fb1e0c6f-0aa5-5b22-b821-9c5cbddd67a7_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 320748,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/335/88ef8b22c6ba46ce8b95d9a47a64cbc1/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-10-11',
 'seats_number': 2,
 'seller_name': None,
 'title': 'CITROËN JUMPY Talla XL BlueHDi 120 S&S 6v Control',
 'transmission': 'manual',
 'trim': 'Talla XL BlueHDi 120 S&S 6v Control',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788875%27 HTTP/1.1" 200 None
2025-12-05 20:02:40 [ayvens.es] INFO: Saving data for 1788875: {'created_time': 1764964960.732823, 'last_price_update_time': 1764964960.732835, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788875') HTTP/1.1" 204 0
2025-12-05 20:02:40 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:40 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:40 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:40 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0df0950>
2025-12-05 20:02:40 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:40 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:40 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1971
2025-12-05 20:02:40 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:41 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:41 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:41 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_dc759675-267d-5217-91f1-b67b3e68f9e4/dc759675-267d-5217-91f1-b67b3e68f9e4_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F251%252F6eebcc6e9ad141c288c23936c9dc67b7%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:41 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/251/6eebcc6e9ad141c288c23936c9dc67b7/images/large/1.jpg saved to cache
2025-12-05 20:02:41 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item dc759675-267d-5217-91f1-b67b3e68f9e4
2025-12-05 20:02:41 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: dc759675-267d-5217-91f1-b67b3e68f9e4, skipping ID generation
2025-12-05 20:02:41 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item dc759675-267d-5217-91f1-b67b3e68f9e4 with scrape type 1
2025-12-05 20:02:41 [crawlers.middlewares.photo_download_middleware] INFO: Removing item dc759675-267d-5217-91f1-b67b3e68f9e4 from tracking dictionaries
2025-12-05 20:02:41 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item dc759675-267d-5217-91f1-b67b3e68f9e4 sending to next pipeline
2025-12-05 20:02:41 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: dc759675-267d-5217-91f1-b67b3e68f9e4, identifier: 1788876
2025-12-05 20:02:41 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/638/e460d359bdce4c87983d063c43b7341a/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:41 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/582/8c9f6bc52ca74f0a909da83ce2b9d2f4/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:41 [ayvens.es] INFO: Found listing with ID: 1788884
2025-12-05 20:02:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788884%27 HTTP/1.1" 200 None
2025-12-05 20:02:41 [ayvens.es] INFO: Scrape type for 1788884: 1
2025-12-05 20:02:41 [ayvens.es] INFO: Extracting data for car ID: 1788884
2025-12-05 20:02:41 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 58bbac59-4e7e-5209-bf06-4d69603c15c7 with identifier: 1788884
2025-12-05 20:02:41 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 58bbac59-4e7e-5209-bf06-4d69603c15c7 with scrape type 1
2025-12-05 20:02:41 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 58bbac59-4e7e-5209-bf06-4d69603c15c7
2025-12-05 20:02:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F541%252Faedb67ebbfa54554948ae0a9e334d6fa%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:41 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/541/aedb67ebbfa54554948ae0a9e334d6fa/images/large/1.jpg not downloaded yet
2025-12-05 20:02:41 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: dc759675-267d-5217-91f1-b67b3e68f9e4 with identifier: 1788876
2025-12-05 20:02:41 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: dc759675-267d-5217-91f1-b67b3e68f9e4 with identifier: 1788876
2025-12-05 20:02:41 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'volkswagen', 'model': 'caddy', 'trim': 'profesional kombi 2.0 tdi scr 4m bmt 122', 'title': 'volkswagen caddy profesional kombi 2.0 tdi scr 4m bmt 122', 'price': 0, 'odometer': 243465, 'odometer_reading_unit': 'km', 'registration_date': '2017-04-28', 'year': 2017, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 4, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '2.0l', 'c_o_2_emission_value': 143, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'dc759675-267d-5217-91f1-b67b3e68f9e4', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_dc759675-267d-5217-91f1-b67b3e68f9e4/dc759675-267d-5217-91f1-b67b3e68f9e4_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/251/6eebcc6e9ad141c288c23936c9dc67b7/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 333355}], 'current_year_diff': 8, 'age_in_months': 103, 'make_model': 'volkswagen caddy', 'sanitized_make': 'volkswagen', 'odometer_in_km': 243465}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 103,
     'auction_title': '',
     'c_o_2_emission_value': 143,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 8,
     'doors_number': 4,
     'engine': '2.0l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': 'dc759675-267d-5217-91f1-b67b3e68f9e4',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'volkswagen',
     'make_model': 'volkswagen caddy',
     'model': 'caddy',
     'odometer': 243465,
     'odometer_in_km': 243465,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_dc759675-267d-5217-91f1-b67b3e68f9e4/dc759675-267d-5217-91f1-b67b3e68f9e4_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 333355,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/251/6eebcc6e9ad141c288c23936c9dc67b7/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2017-04-28',
     'sanitized_make': 'volkswagen',
     'seats_number': 5,
     'seller_name': None,
     'title': 'volkswagen caddy profesional kombi 2.0 tdi scr 4m bmt 122',
     'transmission': 'manual',
     'trim': 'profesional kombi 2.0 tdi scr 4m bmt 122',
     'vin': None,
     'year': 2017}
2025-12-05 20:02:41 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE failed for item: dc759675-267d-5217-91f1-b67b3e68f9e4 spider: ayvens.es
2025-12-05 20:02:41 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: dc759675-267d-5217-91f1-b67b3e68f9e4 due to rule: high_mileage.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'volkswagen', 'model': 'caddy', 'trim': 'profesional kombi 2.0 tdi scr 4m bmt 122', 'title': 'volkswagen caddy profesional kombi 2.0 tdi scr 4m bmt 122', 'price': 0, 'odometer': 243465, 'odometer_reading_unit': 'km', 'registration_date': '2017-04-28', 'year': 2017, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 4, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '2.0l', 'c_o_2_emission_value': 143, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'dc759675-267d-5217-91f1-b67b3e68f9e4', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_dc759675-267d-5217-91f1-b67b3e68f9e4/dc759675-267d-5217-91f1-b67b3e68f9e4_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/251/6eebcc6e9ad141c288c23936c9dc67b7/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 333355}], 'current_year_diff': 8, 'age_in_months': 103, 'make_model': 'volkswagen caddy', 'sanitized_make': 'volkswagen', 'odometer_in_km': 243465}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 103,
     'auction_title': '',
     'c_o_2_emission_value': 143,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 8,
     'doors_number': 4,
     'engine': '2.0l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': 'dc759675-267d-5217-91f1-b67b3e68f9e4',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'volkswagen',
     'make_model': 'volkswagen caddy',
     'model': 'caddy',
     'odometer': 243465,
     'odometer_in_km': 243465,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_dc759675-267d-5217-91f1-b67b3e68f9e4/dc759675-267d-5217-91f1-b67b3e68f9e4_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 333355,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/251/6eebcc6e9ad141c288c23936c9dc67b7/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2017-04-28',
     'sanitized_make': 'volkswagen',
     'seats_number': 5,
     'seller_name': None,
     'title': 'volkswagen caddy profesional kombi 2.0 tdi scr 4m bmt 122',
     'transmission': 'manual',
     'trim': 'profesional kombi 2.0 tdi scr 4m bmt 122',
     'vin': None,
     'year': 2017}
{'auction_title': '',
 'c_o_2_emission_value': 143,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'dc759675-267d-5217-91f1-b67b3e68f9e4',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'CADDY',
 'odometer': 243465,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_dc759675-267d-5217-91f1-b67b3e68f9e4/dc759675-267d-5217-91f1-b67b3e68f9e4_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 333355,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/251/6eebcc6e9ad141c288c23936c9dc67b7/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2017-04-28',
 'seats_number': 5,
 'seller_name': None,
 'title': 'VOLKSWAGEN CADDY Profesional Kombi 2.0 TDI SCR 4M BMT 122',
 'transmission': 'manual',
 'trim': 'Profesional Kombi 2.0 TDI SCR 4M BMT 122',
 'vin': None,
 'year': 2017}
2025-12-05 20:02:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788876%27 HTTP/1.1" 200 None
2025-12-05 20:02:41 [ayvens.es] INFO: Saving data for 1788876: {'created_time': 1764964961.602764, 'last_price_update_time': 1764964961.602774, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788876') HTTP/1.1" 204 0
2025-12-05 20:02:41 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/445/ef6390a1414040699505a24e65cb34b6/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:41 [ayvens.es] INFO: Found listing with ID: 1788885
2025-12-05 20:02:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788885%27 HTTP/1.1" 200 None
2025-12-05 20:02:41 [ayvens.es] INFO: Scrape type for 1788885: 1
2025-12-05 20:02:41 [ayvens.es] INFO: Extracting data for car ID: 1788885
2025-12-05 20:02:41 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 99fc7614-c90a-5445-811f-9eacfc59e1a6 with identifier: 1788885
2025-12-05 20:02:41 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 99fc7614-c90a-5445-811f-9eacfc59e1a6 with scrape type 1
2025-12-05 20:02:41 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 99fc7614-c90a-5445-811f-9eacfc59e1a6
2025-12-05 20:02:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F127%252F69408555f72d4730824601c1f7f816f0%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:41 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/127/69408555f72d4730824601c1f7f816f0/images/large/1.jpg not downloaded yet
2025-12-05 20:02:41 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/383/17aa565583e44433ae3a5d195ec7d239/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:41 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:41 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:41 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:41 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0544550>
2025-12-05 20:02:41 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:41 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:41 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:02:41 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:41 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:41 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:41 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_75df8548-1322-588d-8461-d03ee84a3494/75df8548-1322-588d-8461-d03ee84a3494_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F638%252Fe460d359bdce4c87983d063c43b7341a%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:41 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/638/e460d359bdce4c87983d063c43b7341a/images/large/1.jpg saved to cache
2025-12-05 20:02:41 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 75df8548-1322-588d-8461-d03ee84a3494
2025-12-05 20:02:41 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 75df8548-1322-588d-8461-d03ee84a3494, skipping ID generation
2025-12-05 20:02:41 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 75df8548-1322-588d-8461-d03ee84a3494 with scrape type 1
2025-12-05 20:02:41 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 75df8548-1322-588d-8461-d03ee84a3494 from tracking dictionaries
2025-12-05 20:02:41 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 75df8548-1322-588d-8461-d03ee84a3494 sending to next pipeline
2025-12-05 20:02:42 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 75df8548-1322-588d-8461-d03ee84a3494, identifier: 1788878
2025-12-05 20:02:42 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:42 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:42 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:42 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0463190>
2025-12-05 20:02:42 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:42 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:42 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:02:42 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:42 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:42 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:42 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_95d3b077-c1b2-599b-82ef-98a3d47282dc/95d3b077-c1b2-599b-82ef-98a3d47282dc_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F582%252F8c9f6bc52ca74f0a909da83ce2b9d2f4%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:42 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/582/8c9f6bc52ca74f0a909da83ce2b9d2f4/images/large/1.jpg saved to cache
2025-12-05 20:02:42 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 95d3b077-c1b2-599b-82ef-98a3d47282dc
2025-12-05 20:02:42 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 95d3b077-c1b2-599b-82ef-98a3d47282dc, skipping ID generation
2025-12-05 20:02:42 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 95d3b077-c1b2-599b-82ef-98a3d47282dc with scrape type 1
2025-12-05 20:02:42 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 95d3b077-c1b2-599b-82ef-98a3d47282dc from tracking dictionaries
2025-12-05 20:02:42 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 95d3b077-c1b2-599b-82ef-98a3d47282dc sending to next pipeline
2025-12-05 20:02:42 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 95d3b077-c1b2-599b-82ef-98a3d47282dc, identifier: 1788877
2025-12-05 20:02:42 [ayvens.es] INFO: Found listing with ID: 1788886
2025-12-05 20:02:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788886%27 HTTP/1.1" 200 None
2025-12-05 20:02:42 [ayvens.es] INFO: Scrape type for 1788886: 1
2025-12-05 20:02:42 [ayvens.es] INFO: Extracting data for car ID: 1788886
2025-12-05 20:02:42 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 1ff561e1-8493-534e-bf18-02c28ea15b25 with identifier: 1788886
2025-12-05 20:02:42 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 1ff561e1-8493-534e-bf18-02c28ea15b25 with scrape type 1
2025-12-05 20:02:42 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 1ff561e1-8493-534e-bf18-02c28ea15b25
2025-12-05 20:02:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F103%252Fb282ac2b4cc54c77829c797e11ef90b2%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:42 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/103/b282ac2b4cc54c77829c797e11ef90b2/images/large/1.jpg not downloaded yet
2025-12-05 20:02:42 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 75df8548-1322-588d-8461-d03ee84a3494 with identifier: 1788878
2025-12-05 20:02:42 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 75df8548-1322-588d-8461-d03ee84a3494 with identifier: 1788878
2025-12-05 20:02:42 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: 156052 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    156052
2025-12-05 20:02:42 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE_FOR_COUNTRY failed for item: 75df8548-1322-588d-8461-d03ee84a3494 spider: ayvens.es
2025-12-05 20:02:42 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 75df8548-1322-588d-8461-d03ee84a3494 due to rule: high_mileage_for_country.json, error: 156052 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    156052
{'auction_title': '',
 'c_o_2_emission_value': 121,
 'color': 'Negro',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '75df8548-1322-588d-8461-d03ee84a3494',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': '308',
 'odometer': 156052,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_75df8548-1322-588d-8461-d03ee84a3494/75df8548-1322-588d-8461-d03ee84a3494_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 367973,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/638/e460d359bdce4c87983d063c43b7341a/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-07-28',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT 308 SW Style BlueHDi 130 S&S',
 'transmission': 'manual',
 'trim': 'SW Style BlueHDi 130 S&S',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788878%27 HTTP/1.1" 200 None
2025-12-05 20:02:42 [ayvens.es] INFO: Saving data for 1788878: {'created_time': 1764964962.943277, 'last_price_update_time': 1764964962.943291, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788878') HTTP/1.1" 204 0
2025-12-05 20:02:42 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 95d3b077-c1b2-599b-82ef-98a3d47282dc with identifier: 1788877
2025-12-05 20:02:42 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:02:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:02:42 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 95d3b077-c1b2-599b-82ef-98a3d47282dc with identifier: 1788877
2025-12-05 20:02:43 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 95d3b077-c1b2-599b-82ef-98a3d47282dc with identifier: 1788877 to the API
2025-12-05 20:02:43 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:43 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:43 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/582/8c9f6bc52ca74f0a909da83ce2b9d2f4/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 164,
 'color': 'Cassa',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.6L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '95d3b077-c1b2-599b-82ef-98a3d47282dc',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'KIA',
 'model': 'SPORTAGE',
 'odometer': 83489,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_95d3b077-c1b2-599b-82ef-98a3d47282dc/95d3b077-c1b2-599b-82ef-98a3d47282dc_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 354038,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/582/8c9f6bc52ca74f0a909da83ce2b9d2f4/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-10-25',
 'seats_number': 5,
 'seller_name': None,
 'title': 'KIA SPORTAGE 1.6 MHEV Business 100kW (136CV) 4x4',
 'transmission': 'manual',
 'trim': '1.6 MHEV Business 100kW (136CV) 4x4',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788877%27 HTTP/1.1" 200 None
2025-12-05 20:02:43 [ayvens.es] INFO: Saving data for 1788877: {'created_time': 1764964963.265585, 'last_price_update_time': 1764964963.265596, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788877') HTTP/1.1" 204 0
2025-12-05 20:02:43 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/427/fda6148e6a8746a59d9b7b3b9a20fbda/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:43 [ayvens.es] INFO: Found listing with ID: 1788887
2025-12-05 20:02:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788887%27 HTTP/1.1" 200 None
2025-12-05 20:02:43 [ayvens.es] INFO: Scrape type for 1788887: 1
2025-12-05 20:02:43 [ayvens.es] INFO: Extracting data for car ID: 1788887
2025-12-05 20:02:43 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 87da7f6e-0bb4-5ee7-8d3e-16de00674ea8 with identifier: 1788887
2025-12-05 20:02:43 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 87da7f6e-0bb4-5ee7-8d3e-16de00674ea8 with scrape type 1
2025-12-05 20:02:43 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 87da7f6e-0bb4-5ee7-8d3e-16de00674ea8
2025-12-05 20:02:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F449%252F91a762a26b3842c5ac9e27ff8f9b6ebf%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:43 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/449/91a762a26b3842c5ac9e27ff8f9b6ebf/images/large/1.jpg not downloaded yet
2025-12-05 20:02:43 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:43 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:43 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:43 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0d5f250>
2025-12-05 20:02:43 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:43 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:43 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:02:43 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:43 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:43 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:43 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_eab09ec2-d9f2-5748-9650-a123712d465a/eab09ec2-d9f2-5748-9650-a123712d465a_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F445%252Fef6390a1414040699505a24e65cb34b6%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:43 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/445/ef6390a1414040699505a24e65cb34b6/images/large/1.jpg saved to cache
2025-12-05 20:02:43 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item eab09ec2-d9f2-5748-9650-a123712d465a
2025-12-05 20:02:43 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: eab09ec2-d9f2-5748-9650-a123712d465a, skipping ID generation
2025-12-05 20:02:43 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item eab09ec2-d9f2-5748-9650-a123712d465a with scrape type 1
2025-12-05 20:02:43 [crawlers.middlewares.photo_download_middleware] INFO: Removing item eab09ec2-d9f2-5748-9650-a123712d465a from tracking dictionaries
2025-12-05 20:02:43 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item eab09ec2-d9f2-5748-9650-a123712d465a sending to next pipeline
2025-12-05 20:02:43 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: eab09ec2-d9f2-5748-9650-a123712d465a, identifier: 1788880
2025-12-05 20:02:43 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:43 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:43 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:43 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0c793d0>
2025-12-05 20:02:43 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:43 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:43 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:02:43 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:44 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:44 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:44 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_aebb980e-1715-5138-a761-3e49ade1d6a1/aebb980e-1715-5138-a761-3e49ade1d6a1_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F383%252F17aa565583e44433ae3a5d195ec7d239%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:44 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/383/17aa565583e44433ae3a5d195ec7d239/images/large/1.jpg saved to cache
2025-12-05 20:02:44 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item aebb980e-1715-5138-a761-3e49ade1d6a1
2025-12-05 20:02:44 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: aebb980e-1715-5138-a761-3e49ade1d6a1, skipping ID generation
2025-12-05 20:02:44 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item aebb980e-1715-5138-a761-3e49ade1d6a1 with scrape type 1
2025-12-05 20:02:44 [crawlers.middlewares.photo_download_middleware] INFO: Removing item aebb980e-1715-5138-a761-3e49ade1d6a1 from tracking dictionaries
2025-12-05 20:02:44 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item aebb980e-1715-5138-a761-3e49ade1d6a1 sending to next pipeline
2025-12-05 20:02:44 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: aebb980e-1715-5138-a761-3e49ade1d6a1, identifier: 1788879
2025-12-05 20:02:44 [ayvens.es] INFO: Found listing with ID: 1788888
2025-12-05 20:02:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788888%27 HTTP/1.1" 200 None
2025-12-05 20:02:44 [ayvens.es] INFO: Scrape type for 1788888: 1
2025-12-05 20:02:44 [ayvens.es] INFO: Extracting data for car ID: 1788888
2025-12-05 20:02:44 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 78d7b5ea-91eb-5003-9995-d33fc807f4ae with identifier: 1788888
2025-12-05 20:02:44 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 78d7b5ea-91eb-5003-9995-d33fc807f4ae with scrape type 1
2025-12-05 20:02:44 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 78d7b5ea-91eb-5003-9995-d33fc807f4ae
2025-12-05 20:02:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F229%252F9ec3c358721d4291aeb2d1e127b5ca4d%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:44 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/229/9ec3c358721d4291aeb2d1e127b5ca4d/images/large/1.jpg not downloaded yet
2025-12-05 20:02:44 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: eab09ec2-d9f2-5748-9650-a123712d465a with identifier: 1788880
2025-12-05 20:02:44 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: eab09ec2-d9f2-5748-9650-a123712d465a with identifier: 1788880
2025-12-05 20:02:44 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'azul', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'opel', 'model': 'insignia', 'trim': 'gs 1.6 cdti 100kw turbo d selec pro', 'title': 'opel insignia gs 1.6 cdti 100kw turbo d selec pro', 'price': 0, 'odometer': 238171, 'odometer_reading_unit': 'km', 'registration_date': '2019-03-29', 'year': 2019, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.6l', 'c_o_2_emission_value': 124, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'eab09ec2-d9f2-5748-9650-a123712d465a', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_eab09ec2-d9f2-5748-9650-a123712d465a/eab09ec2-d9f2-5748-9650-a123712d465a_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/445/ef6390a1414040699505a24e65cb34b6/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 285966}], 'current_year_diff': 6, 'age_in_months': 80, 'make_model': 'opel insignia', 'sanitized_make': 'opel', 'odometer_in_km': 238171}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 80,
     'auction_title': '',
     'c_o_2_emission_value': 124,
     'color': 'azul',
     'currency': 'eur',
     'current_year_diff': 6,
     'doors_number': 5,
     'engine': '1.6l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': 'eab09ec2-d9f2-5748-9650-a123712d465a',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'opel',
     'make_model': 'opel insignia',
     'model': 'insignia',
     'odometer': 238171,
     'odometer_in_km': 238171,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_eab09ec2-d9f2-5748-9650-a123712d465a/eab09ec2-d9f2-5748-9650-a123712d465a_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 285966,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/445/ef6390a1414040699505a24e65cb34b6/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2019-03-29',
     'sanitized_make': 'opel',
     'seats_number': 5,
     'seller_name': None,
     'title': 'opel insignia gs 1.6 cdti 100kw turbo d selec pro',
     'transmission': 'manual',
     'trim': 'gs 1.6 cdti 100kw turbo d selec pro',
     'vin': None,
     'year': 2019}
2025-12-05 20:02:44 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE failed for item: eab09ec2-d9f2-5748-9650-a123712d465a spider: ayvens.es
2025-12-05 20:02:44 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: eab09ec2-d9f2-5748-9650-a123712d465a due to rule: high_mileage.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'azul', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'opel', 'model': 'insignia', 'trim': 'gs 1.6 cdti 100kw turbo d selec pro', 'title': 'opel insignia gs 1.6 cdti 100kw turbo d selec pro', 'price': 0, 'odometer': 238171, 'odometer_reading_unit': 'km', 'registration_date': '2019-03-29', 'year': 2019, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.6l', 'c_o_2_emission_value': 124, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'eab09ec2-d9f2-5748-9650-a123712d465a', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_eab09ec2-d9f2-5748-9650-a123712d465a/eab09ec2-d9f2-5748-9650-a123712d465a_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/445/ef6390a1414040699505a24e65cb34b6/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 285966}], 'current_year_diff': 6, 'age_in_months': 80, 'make_model': 'opel insignia', 'sanitized_make': 'opel', 'odometer_in_km': 238171}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 80,
     'auction_title': '',
     'c_o_2_emission_value': 124,
     'color': 'azul',
     'currency': 'eur',
     'current_year_diff': 6,
     'doors_number': 5,
     'engine': '1.6l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': 'eab09ec2-d9f2-5748-9650-a123712d465a',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'opel',
     'make_model': 'opel insignia',
     'model': 'insignia',
     'odometer': 238171,
     'odometer_in_km': 238171,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_eab09ec2-d9f2-5748-9650-a123712d465a/eab09ec2-d9f2-5748-9650-a123712d465a_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 285966,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/445/ef6390a1414040699505a24e65cb34b6/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2019-03-29',
     'sanitized_make': 'opel',
     'seats_number': 5,
     'seller_name': None,
     'title': 'opel insignia gs 1.6 cdti 100kw turbo d selec pro',
     'transmission': 'manual',
     'trim': 'gs 1.6 cdti 100kw turbo d selec pro',
     'vin': None,
     'year': 2019}
{'auction_title': '',
 'c_o_2_emission_value': 124,
 'color': 'Azul',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.6L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'eab09ec2-d9f2-5748-9650-a123712d465a',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'OPEL',
 'model': 'INSIGNIA',
 'odometer': 238171,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_eab09ec2-d9f2-5748-9650-a123712d465a/eab09ec2-d9f2-5748-9650-a123712d465a_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 285966,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/445/ef6390a1414040699505a24e65cb34b6/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-03-29',
 'seats_number': 5,
 'seller_name': None,
 'title': 'OPEL INSIGNIA GS 1.6 CDTi 100kW Turbo D Selec Pro',
 'transmission': 'manual',
 'trim': 'GS 1.6 CDTi 100kW Turbo D Selec Pro',
 'vin': None,
 'year': 2019}
2025-12-05 20:02:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788880%27 HTTP/1.1" 200 None
2025-12-05 20:02:44 [ayvens.es] INFO: Saving data for 1788880: {'created_time': 1764964964.52672, 'last_price_update_time': 1764964964.526731, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788880') HTTP/1.1" 204 0
2025-12-05 20:02:44 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: aebb980e-1715-5138-a761-3e49ade1d6a1 with identifier: 1788879
2025-12-05 20:02:44 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: aebb980e-1715-5138-a761-3e49ade1d6a1 with identifier: 1788879
2025-12-05 20:02:44 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: aebb980e-1715-5138-a761-3e49ade1d6a1 with identifier: 1788879 to the API
2025-12-05 20:02:44 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:44 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/383/17aa565583e44433ae3a5d195ec7d239/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 119,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'aebb980e-1715-5138-a761-3e49ade1d6a1',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'AUDI',
 'model': 'Q3',
 'odometer': 131208,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_aebb980e-1715-5138-a761-3e49ade1d6a1/aebb980e-1715-5138-a761-3e49ade1d6a1_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 331278,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/383/17aa565583e44433ae3a5d195ec7d239/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2018-08-27',
 'seats_number': 5,
 'seller_name': None,
 'title': 'AUDI Q3 Sport edition 2.0 TDI 110kW (150CV)',
 'transmission': 'manual',
 'trim': 'Sport edition 2.0 TDI 110kW (150CV)',
 'vin': None,
 'year': 2018}
2025-12-05 20:02:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788879%27 HTTP/1.1" 200 None
2025-12-05 20:02:44 [ayvens.es] INFO: Saving data for 1788879: {'created_time': 1764964964.883693, 'last_price_update_time': 1764964964.883704, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788879') HTTP/1.1" 204 0
2025-12-05 20:02:44 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/105/8d338a1beec749a0893038782695b337/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:44 [ayvens.es] INFO: Found listing with ID: 1788889
2025-12-05 20:02:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788889%27 HTTP/1.1" 200 None
2025-12-05 20:02:44 [ayvens.es] INFO: Scrape type for 1788889: 1
2025-12-05 20:02:44 [ayvens.es] INFO: Extracting data for car ID: 1788889
2025-12-05 20:02:44 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 5cd1b8f7-8fb5-528c-afd0-1605c9143822 with identifier: 1788889
2025-12-05 20:02:44 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 5cd1b8f7-8fb5-528c-afd0-1605c9143822 with scrape type 1
2025-12-05 20:02:44 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 5cd1b8f7-8fb5-528c-afd0-1605c9143822
2025-12-05 20:02:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F168%252F81e659889a5649809f06902517896947%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:44 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/168/81e659889a5649809f06902517896947/images/large/1.jpg not downloaded yet
2025-12-05 20:02:44 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/845/454fb9c25db84227bb23de473454048c/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:44 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/541/aedb67ebbfa54554948ae0a9e334d6fa/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:44 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:44 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:44 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:44 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b84cd0>
2025-12-05 20:02:44 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:44 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:45 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:02:45 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:45 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:45 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:45 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_f7186b82-b603-5803-8dc8-4be83117e4d6/f7186b82-b603-5803-8dc8-4be83117e4d6_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F427%252Ffda6148e6a8746a59d9b7b3b9a20fbda%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:45 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/427/fda6148e6a8746a59d9b7b3b9a20fbda/images/large/1.jpg saved to cache
2025-12-05 20:02:45 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item f7186b82-b603-5803-8dc8-4be83117e4d6
2025-12-05 20:02:45 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: f7186b82-b603-5803-8dc8-4be83117e4d6, skipping ID generation
2025-12-05 20:02:45 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f7186b82-b603-5803-8dc8-4be83117e4d6 with scrape type 1
2025-12-05 20:02:45 [crawlers.middlewares.photo_download_middleware] INFO: Removing item f7186b82-b603-5803-8dc8-4be83117e4d6 from tracking dictionaries
2025-12-05 20:02:45 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item f7186b82-b603-5803-8dc8-4be83117e4d6 sending to next pipeline
2025-12-05 20:02:45 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: f7186b82-b603-5803-8dc8-4be83117e4d6, identifier: 1788881
2025-12-05 20:02:45 [ayvens.es] INFO: Found listing with ID: 1788890
2025-12-05 20:02:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788890%27 HTTP/1.1" 200 None
2025-12-05 20:02:45 [ayvens.es] INFO: Scrape type for 1788890: 1
2025-12-05 20:02:45 [ayvens.es] INFO: Extracting data for car ID: 1788890
2025-12-05 20:02:45 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: c8b92838-cc17-5c2f-acb6-1b649dcab627 with identifier: 1788890
2025-12-05 20:02:45 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item c8b92838-cc17-5c2f-acb6-1b649dcab627 with scrape type 1
2025-12-05 20:02:45 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item c8b92838-cc17-5c2f-acb6-1b649dcab627
2025-12-05 20:02:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F082%252Fe3c1b10e6b7f4612bfc76b5858516ad5%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:45 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/082/e3c1b10e6b7f4612bfc76b5858516ad5/images/large/1.jpg not downloaded yet
2025-12-05 20:02:45 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: f7186b82-b603-5803-8dc8-4be83117e4d6 with identifier: 1788881
2025-12-05 20:02:45 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: f7186b82-b603-5803-8dc8-4be83117e4d6 with identifier: 1788881
2025-12-05 20:02:45 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: f7186b82-b603-5803-8dc8-4be83117e4d6 with identifier: 1788881 to the API
2025-12-05 20:02:45 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:45 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/427/fda6148e6a8746a59d9b7b3b9a20fbda/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 86,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.8L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'hybrid',
 'id': 'f7186b82-b603-5803-8dc8-4be83117e4d6',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'TOYOTA',
 'model': 'C-HR',
 'odometer': 59055,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_f7186b82-b603-5803-8dc8-4be83117e4d6/f7186b82-b603-5803-8dc8-4be83117e4d6_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 358566,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/427/fda6148e6a8746a59d9b7b3b9a20fbda/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-04-24',
 'seats_number': 5,
 'seller_name': None,
 'title': 'TOYOTA C-HR 1.8 125H Advance',
 'transmission': 'manual',
 'trim': '1.8 125H Advance',
 'vin': None,
 'year': 2019}
2025-12-05 20:02:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788881%27 HTTP/1.1" 200 None
2025-12-05 20:02:45 [ayvens.es] INFO: Saving data for 1788881: {'created_time': 1764964965.736845, 'last_price_update_time': 1764964965.736856, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788881') HTTP/1.1" 204 0
2025-12-05 20:02:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/127/69408555f72d4730824601c1f7f816f0/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:45 [ayvens.es] INFO: Found listing with ID: 1788891
2025-12-05 20:02:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788891%27 HTTP/1.1" 200 None
2025-12-05 20:02:45 [ayvens.es] INFO: Scrape type for 1788891: 1
2025-12-05 20:02:45 [ayvens.es] INFO: Extracting data for car ID: 1788891
2025-12-05 20:02:45 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: e01808eb-4d99-5c8d-8386-5c90a3b8cbbd with identifier: 1788891
2025-12-05 20:02:45 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e01808eb-4d99-5c8d-8386-5c90a3b8cbbd with scrape type 1
2025-12-05 20:02:45 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item e01808eb-4d99-5c8d-8386-5c90a3b8cbbd
2025-12-05 20:02:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F068%252F9da01439420d44c6bb5669f33353cee4%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:45 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/068/9da01439420d44c6bb5669f33353cee4/images/large/1.jpg not downloaded yet
2025-12-05 20:02:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/103/b282ac2b4cc54c77829c797e11ef90b2/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:45 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:45 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:45 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:45 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0478890>
2025-12-05 20:02:45 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:45 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:46 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:02:46 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:46 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:46 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:46 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_caa8d3ba-c53d-5300-922d-0ad7a28c1449/caa8d3ba-c53d-5300-922d-0ad7a28c1449_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F105%252F8d338a1beec749a0893038782695b337%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:46 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/105/8d338a1beec749a0893038782695b337/images/large/1.jpg saved to cache
2025-12-05 20:02:46 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item caa8d3ba-c53d-5300-922d-0ad7a28c1449
2025-12-05 20:02:46 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: caa8d3ba-c53d-5300-922d-0ad7a28c1449, skipping ID generation
2025-12-05 20:02:46 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item caa8d3ba-c53d-5300-922d-0ad7a28c1449 with scrape type 1
2025-12-05 20:02:46 [crawlers.middlewares.photo_download_middleware] INFO: Removing item caa8d3ba-c53d-5300-922d-0ad7a28c1449 from tracking dictionaries
2025-12-05 20:02:46 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item caa8d3ba-c53d-5300-922d-0ad7a28c1449 sending to next pipeline
2025-12-05 20:02:46 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: caa8d3ba-c53d-5300-922d-0ad7a28c1449, identifier: 1788883
2025-12-05 20:02:46 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:46 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:46 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:46 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b459d0>
2025-12-05 20:02:46 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:46 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:46 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:02:46 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:46 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:46 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:46 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_ec7f1733-6376-594d-8eaa-74bf1b70d714/ec7f1733-6376-594d-8eaa-74bf1b70d714_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F845%252F454fb9c25db84227bb23de473454048c%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:46 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/845/454fb9c25db84227bb23de473454048c/images/large/1.jpg saved to cache
2025-12-05 20:02:46 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item ec7f1733-6376-594d-8eaa-74bf1b70d714
2025-12-05 20:02:46 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: ec7f1733-6376-594d-8eaa-74bf1b70d714, skipping ID generation
2025-12-05 20:02:46 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ec7f1733-6376-594d-8eaa-74bf1b70d714 with scrape type 1
2025-12-05 20:02:46 [crawlers.middlewares.photo_download_middleware] INFO: Removing item ec7f1733-6376-594d-8eaa-74bf1b70d714 from tracking dictionaries
2025-12-05 20:02:46 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item ec7f1733-6376-594d-8eaa-74bf1b70d714 sending to next pipeline
2025-12-05 20:02:46 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: ec7f1733-6376-594d-8eaa-74bf1b70d714, identifier: 1788882
2025-12-05 20:02:46 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:46 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:46 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:46 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b80650>
2025-12-05 20:02:46 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:46 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:46 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:02:46 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:46 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:46 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:47 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_58bbac59-4e7e-5209-bf06-4d69603c15c7/58bbac59-4e7e-5209-bf06-4d69603c15c7_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F541%252Faedb67ebbfa54554948ae0a9e334d6fa%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:47 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/541/aedb67ebbfa54554948ae0a9e334d6fa/images/large/1.jpg saved to cache
2025-12-05 20:02:47 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 58bbac59-4e7e-5209-bf06-4d69603c15c7
2025-12-05 20:02:47 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 58bbac59-4e7e-5209-bf06-4d69603c15c7, skipping ID generation
2025-12-05 20:02:47 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 58bbac59-4e7e-5209-bf06-4d69603c15c7 with scrape type 1
2025-12-05 20:02:47 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 58bbac59-4e7e-5209-bf06-4d69603c15c7 from tracking dictionaries
2025-12-05 20:02:47 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 58bbac59-4e7e-5209-bf06-4d69603c15c7 sending to next pipeline
2025-12-05 20:02:47 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 58bbac59-4e7e-5209-bf06-4d69603c15c7, identifier: 1788884
2025-12-05 20:02:47 [ayvens.es] INFO: Found listing with ID: 1788892
2025-12-05 20:02:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788892%27 HTTP/1.1" 200 None
2025-12-05 20:02:47 [ayvens.es] INFO: Scrape type for 1788892: 1
2025-12-05 20:02:47 [ayvens.es] INFO: Extracting data for car ID: 1788892
2025-12-05 20:02:47 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 7d85304b-0ea0-5784-a1ee-bb0fdb7cde36 with identifier: 1788892
2025-12-05 20:02:47 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 7d85304b-0ea0-5784-a1ee-bb0fdb7cde36 with scrape type 1
2025-12-05 20:02:47 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 7d85304b-0ea0-5784-a1ee-bb0fdb7cde36
2025-12-05 20:02:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F003%252F74b5a866d96f4062a2e419d1803fd148%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:47 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/003/74b5a866d96f4062a2e419d1803fd148/images/large/1.jpg not downloaded yet
2025-12-05 20:02:47 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: caa8d3ba-c53d-5300-922d-0ad7a28c1449 with identifier: 1788883
2025-12-05 20:02:47 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: caa8d3ba-c53d-5300-922d-0ad7a28c1449 with identifier: 1788883
2025-12-05 20:02:47 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: caa8d3ba-c53d-5300-922d-0ad7a28c1449 with identifier: 1788883 to the API
2025-12-05 20:02:47 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:47 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/105/8d338a1beec749a0893038782695b337/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 120,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'caa8d3ba-c53d-5300-922d-0ad7a28c1449',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'CITROËN',
 'model': 'C3 AIRCROSS',
 'odometer': 82563,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_caa8d3ba-c53d-5300-922d-0ad7a28c1449/caa8d3ba-c53d-5300-922d-0ad7a28c1449_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 338316,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/105/8d338a1beec749a0893038782695b337/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2022-11-10',
 'seats_number': 5,
 'seller_name': None,
 'title': 'CITROËN C3 AIRCROSS BlueHDi 81kW (110CV) S&S Feel Pack',
 'transmission': 'manual',
 'trim': 'BlueHDi 81kW (110CV) S&S Feel Pack',
 'vin': None,
 'year': 2022}
2025-12-05 20:02:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788883%27 HTTP/1.1" 200 None
2025-12-05 20:02:47 [ayvens.es] INFO: Saving data for 1788883: {'created_time': 1764964967.584313, 'last_price_update_time': 1764964967.584324, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788883') HTTP/1.1" 204 0
2025-12-05 20:02:47 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: ec7f1733-6376-594d-8eaa-74bf1b70d714 with identifier: 1788882
2025-12-05 20:02:47 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: ec7f1733-6376-594d-8eaa-74bf1b70d714 with identifier: 1788882
2025-12-05 20:02:47 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'opel' containing 'opel' and model 'combo' containing 'combo'
2025-12-05 20:02:47 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: ec7f1733-6376-594d-8eaa-74bf1b70d714 spider: ayvens.es
2025-12-05 20:02:47 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: ec7f1733-6376-594d-8eaa-74bf1b70d714 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'opel' containing 'opel' and model 'combo' containing 'combo'
{'auction_title': '',
 'c_o_2_emission_value': 135,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'ec7f1733-6376-594d-8eaa-74bf1b70d714',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'OPEL',
 'model': 'COMBO',
 'odometer': 135915,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ec7f1733-6376-594d-8eaa-74bf1b70d714/ec7f1733-6376-594d-8eaa-74bf1b70d714_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 594722,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/845/454fb9c25db84227bb23de473454048c/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-06-08',
 'seats_number': 5,
 'seller_name': None,
 'title': 'OPEL COMBO 1.5 TD 75kW (100CV) S/S Edition Plus L',
 'transmission': 'manual',
 'trim': '1.5 TD 75kW (100CV) S/S Edition Plus L',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788882%27 HTTP/1.1" 200 None
2025-12-05 20:02:47 [ayvens.es] INFO: Saving data for 1788882: {'created_time': 1764964967.844857, 'last_price_update_time': 1764964967.844866, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788882') HTTP/1.1" 204 0
2025-12-05 20:02:47 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 58bbac59-4e7e-5209-bf06-4d69603c15c7 with identifier: 1788884
2025-12-05 20:02:47 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 58bbac59-4e7e-5209-bf06-4d69603c15c7 with identifier: 1788884
2025-12-05 20:02:47 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: 165096 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    165096
2025-12-05 20:02:47 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE_FOR_COUNTRY failed for item: 58bbac59-4e7e-5209-bf06-4d69603c15c7 spider: ayvens.es
2025-12-05 20:02:48 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 58bbac59-4e7e-5209-bf06-4d69603c15c7 due to rule: high_mileage_for_country.json, error: 165096 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    165096
{'auction_title': '',
 'c_o_2_emission_value': 148,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '58bbac59-4e7e-5209-bf06-4d69603c15c7',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SKODA',
 'model': 'KAROQ',
 'odometer': 165096,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_58bbac59-4e7e-5209-bf06-4d69603c15c7/58bbac59-4e7e-5209-bf06-4d69603c15c7_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 358705,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/541/aedb67ebbfa54554948ae0a9e334d6fa/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-04-28',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SKODA KAROQ 2.0 TDI 110kW (150CV) DSG 4X4 Ambition',
 'transmission': 'manual',
 'trim': '2.0 TDI 110kW (150CV) DSG 4X4 Ambition',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788884%27 HTTP/1.1" 200 None
2025-12-05 20:02:48 [ayvens.es] INFO: Saving data for 1788884: {'created_time': 1764964968.124389, 'last_price_update_time': 1764964968.124402, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788884') HTTP/1.1" 204 0
2025-12-05 20:02:48 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/449/91a762a26b3842c5ac9e27ff8f9b6ebf/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:48 [ayvens.es] INFO: Found listing with ID: 1788893
2025-12-05 20:02:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788893%27 HTTP/1.1" 200 None
2025-12-05 20:02:48 [ayvens.es] INFO: Scrape type for 1788893: 1
2025-12-05 20:02:48 [ayvens.es] INFO: Extracting data for car ID: 1788893
2025-12-05 20:02:48 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 824142ed-f7e4-512e-b5ef-32d753cf3a9a with identifier: 1788893
2025-12-05 20:02:48 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 824142ed-f7e4-512e-b5ef-32d753cf3a9a with scrape type 1
2025-12-05 20:02:48 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 824142ed-f7e4-512e-b5ef-32d753cf3a9a
2025-12-05 20:02:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F846%252F8819adfe2fd54957ba4fc4c15f2d490b%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:48 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/846/8819adfe2fd54957ba4fc4c15f2d490b/images/large/1.jpg not downloaded yet
2025-12-05 20:02:48 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:48 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:48 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:48 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0442890>
2025-12-05 20:02:48 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:48 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:48 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:02:48 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:48 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:48 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:48 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_99fc7614-c90a-5445-811f-9eacfc59e1a6/99fc7614-c90a-5445-811f-9eacfc59e1a6_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F127%252F69408555f72d4730824601c1f7f816f0%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:48 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/127/69408555f72d4730824601c1f7f816f0/images/large/1.jpg saved to cache
2025-12-05 20:02:48 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 99fc7614-c90a-5445-811f-9eacfc59e1a6
2025-12-05 20:02:48 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 99fc7614-c90a-5445-811f-9eacfc59e1a6, skipping ID generation
2025-12-05 20:02:48 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 99fc7614-c90a-5445-811f-9eacfc59e1a6 with scrape type 1
2025-12-05 20:02:48 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 99fc7614-c90a-5445-811f-9eacfc59e1a6 from tracking dictionaries
2025-12-05 20:02:48 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 99fc7614-c90a-5445-811f-9eacfc59e1a6 sending to next pipeline
2025-12-05 20:02:48 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 99fc7614-c90a-5445-811f-9eacfc59e1a6, identifier: 1788885
2025-12-05 20:02:48 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:48 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:48 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:48 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b5a510>
2025-12-05 20:02:48 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:48 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:48 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:02:48 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:49 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:49 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:49 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_1ff561e1-8493-534e-bf18-02c28ea15b25/1ff561e1-8493-534e-bf18-02c28ea15b25_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F103%252Fb282ac2b4cc54c77829c797e11ef90b2%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:49 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/103/b282ac2b4cc54c77829c797e11ef90b2/images/large/1.jpg saved to cache
2025-12-05 20:02:49 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 1ff561e1-8493-534e-bf18-02c28ea15b25
2025-12-05 20:02:49 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 1ff561e1-8493-534e-bf18-02c28ea15b25, skipping ID generation
2025-12-05 20:02:49 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 1ff561e1-8493-534e-bf18-02c28ea15b25 with scrape type 1
2025-12-05 20:02:49 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 1ff561e1-8493-534e-bf18-02c28ea15b25 from tracking dictionaries
2025-12-05 20:02:49 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 1ff561e1-8493-534e-bf18-02c28ea15b25 sending to next pipeline
2025-12-05 20:02:49 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 1ff561e1-8493-534e-bf18-02c28ea15b25, identifier: 1788886
2025-12-05 20:02:49 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 99fc7614-c90a-5445-811f-9eacfc59e1a6 with identifier: 1788885
2025-12-05 20:02:49 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 99fc7614-c90a-5445-811f-9eacfc59e1a6 with identifier: 1788885
2025-12-05 20:02:49 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 99fc7614-c90a-5445-811f-9eacfc59e1a6 with identifier: 1788885 to the API
2025-12-05 20:02:49 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:49 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:49 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/127/69408555f72d4730824601c1f7f816f0/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 162,
 'color': 'Negro',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '99fc7614-c90a-5445-811f-9eacfc59e1a6',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'MERCEDES-BENZ',
 'model': 'GLC COUPE',
 'odometer': 117128,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_99fc7614-c90a-5445-811f-9eacfc59e1a6/99fc7614-c90a-5445-811f-9eacfc59e1a6_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 358119,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/127/69408555f72d4730824601c1f7f816f0/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-04-13',
 'seats_number': 5,
 'seller_name': None,
 'title': 'MERCEDES-BENZ GLC COUPE GLC 300 4MATIC',
 'transmission': 'manual',
 'trim': 'GLC 300 4MATIC',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788885%27 HTTP/1.1" 200 None
2025-12-05 20:02:49 [ayvens.es] INFO: Saving data for 1788885: {'created_time': 1764964969.594122, 'last_price_update_time': 1764964969.594133, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788885') HTTP/1.1" 204 0
2025-12-05 20:02:49 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 1ff561e1-8493-534e-bf18-02c28ea15b25 with identifier: 1788886
2025-12-05 20:02:49 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 1ff561e1-8493-534e-bf18-02c28ea15b25 with identifier: 1788886
2025-12-05 20:02:49 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 1ff561e1-8493-534e-bf18-02c28ea15b25 with identifier: 1788886 to the API
2025-12-05 20:02:49 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:49 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:50 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/103/b282ac2b4cc54c77829c797e11ef90b2/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 135,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.3L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '1ff561e1-8493-534e-bf18-02c28ea15b25',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'MERCEDES-BENZ',
 'model': 'A-CLASS',
 'odometer': 15359,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_1ff561e1-8493-534e-bf18-02c28ea15b25/1ff561e1-8493-534e-bf18-02c28ea15b25_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 374344,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/103/b282ac2b4cc54c77829c797e11ef90b2/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-11-11',
 'seats_number': 5,
 'seller_name': None,
 'title': 'MERCEDES-BENZ A-CLASS A 200',
 'transmission': 'manual',
 'trim': 'A 200',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788886%27 HTTP/1.1" 200 None
2025-12-05 20:02:50 [ayvens.es] INFO: Saving data for 1788886: {'created_time': 1764964970.072875, 'last_price_update_time': 1764964970.072884, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788886') HTTP/1.1" 204 0
2025-12-05 20:02:50 [ayvens.es] INFO: Found listing with ID: 1788894
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788894%27 HTTP/1.1" 200 None
2025-12-05 20:02:50 [ayvens.es] INFO: Scrape type for 1788894: 1
2025-12-05 20:02:50 [ayvens.es] INFO: Extracting data for car ID: 1788894
2025-12-05 20:02:50 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 04137861-8d1b-5ef9-a869-87207ac8e377 with identifier: 1788894
2025-12-05 20:02:50 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 04137861-8d1b-5ef9-a869-87207ac8e377 with scrape type 1
2025-12-05 20:02:50 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 04137861-8d1b-5ef9-a869-87207ac8e377
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F343%252F31f78186538c486daeccc46697cd1a0c%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:50 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/343/31f78186538c486daeccc46697cd1a0c/images/large/1.jpg not downloaded yet
2025-12-05 20:02:50 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/168/81e659889a5649809f06902517896947/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:50 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/229/9ec3c358721d4291aeb2d1e127b5ca4d/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:50 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:50 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:50 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:50 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0aa4910>
2025-12-05 20:02:50 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:50 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_87da7f6e-0bb4-5ee7-8d3e-16de00674ea8/87da7f6e-0bb4-5ee7-8d3e-16de00674ea8_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F449%252F91a762a26b3842c5ac9e27ff8f9b6ebf%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:50 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/449/91a762a26b3842c5ac9e27ff8f9b6ebf/images/large/1.jpg saved to cache
2025-12-05 20:02:50 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 87da7f6e-0bb4-5ee7-8d3e-16de00674ea8
2025-12-05 20:02:50 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 87da7f6e-0bb4-5ee7-8d3e-16de00674ea8, skipping ID generation
2025-12-05 20:02:50 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 87da7f6e-0bb4-5ee7-8d3e-16de00674ea8 with scrape type 1
2025-12-05 20:02:50 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 87da7f6e-0bb4-5ee7-8d3e-16de00674ea8 from tracking dictionaries
2025-12-05 20:02:50 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 87da7f6e-0bb4-5ee7-8d3e-16de00674ea8 sending to next pipeline
2025-12-05 20:02:50 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 87da7f6e-0bb4-5ee7-8d3e-16de00674ea8, identifier: 1788887
2025-12-05 20:02:50 [ayvens.es] INFO: Found listing with ID: 1788895
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788895%27 HTTP/1.1" 200 None
2025-12-05 20:02:50 [ayvens.es] INFO: Scrape type for 1788895: 1
2025-12-05 20:02:50 [ayvens.es] INFO: Extracting data for car ID: 1788895
2025-12-05 20:02:50 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 44579e26-a161-50b0-b1e1-d2efe0176205 with identifier: 1788895
2025-12-05 20:02:50 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 44579e26-a161-50b0-b1e1-d2efe0176205 with scrape type 1
2025-12-05 20:02:50 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 44579e26-a161-50b0-b1e1-d2efe0176205
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F805%252F2d8594d13f9c4439b78745db43f691ac%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:50 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/805/2d8594d13f9c4439b78745db43f691ac/images/large/1.jpg not downloaded yet
2025-12-05 20:02:50 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 87da7f6e-0bb4-5ee7-8d3e-16de00674ea8 with identifier: 1788887
2025-12-05 20:02:50 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 87da7f6e-0bb4-5ee7-8d3e-16de00674ea8 with identifier: 1788887
2025-12-05 20:02:50 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'negro', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'volkswagen', 'model': 'golf', 'trim': 'advance 1.6 tdi 110cv bmt', 'title': 'volkswagen golf advance 1.6 tdi 110cv bmt', 'price': 0, 'odometer': 409160, 'odometer_reading_unit': 'km', 'registration_date': '2016-10-19', 'year': 2016, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.6l', 'c_o_2_emission_value': 99, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '87da7f6e-0bb4-5ee7-8d3e-16de00674ea8', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_87da7f6e-0bb4-5ee7-8d3e-16de00674ea8/87da7f6e-0bb4-5ee7-8d3e-16de00674ea8_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/449/91a762a26b3842c5ac9e27ff8f9b6ebf/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 377019}], 'current_year_diff': 9, 'age_in_months': 109, 'make_model': 'volkswagen golf', 'sanitized_make': 'volkswagen', 'odometer_in_km': 409160}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 109,
     'auction_title': '',
     'c_o_2_emission_value': 99,
     'color': 'negro',
     'currency': 'eur',
     'current_year_diff': 9,
     'doors_number': 5,
     'engine': '1.6l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': '87da7f6e-0bb4-5ee7-8d3e-16de00674ea8',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'volkswagen',
     'make_model': 'volkswagen golf',
     'model': 'golf',
     'odometer': 409160,
     'odometer_in_km': 409160,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_87da7f6e-0bb4-5ee7-8d3e-16de00674ea8/87da7f6e-0bb4-5ee7-8d3e-16de00674ea8_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 377019,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/449/91a762a26b3842c5ac9e27ff8f9b6ebf/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2016-10-19',
     'sanitized_make': 'volkswagen',
     'seats_number': 5,
     'seller_name': None,
     'title': 'volkswagen golf advance 1.6 tdi 110cv bmt',
     'transmission': 'manual',
     'trim': 'advance 1.6 tdi 110cv bmt',
     'vin': None,
     'year': 2016}
2025-12-05 20:02:50 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE failed for item: 87da7f6e-0bb4-5ee7-8d3e-16de00674ea8 spider: ayvens.es
2025-12-05 20:02:50 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 87da7f6e-0bb4-5ee7-8d3e-16de00674ea8 due to rule: high_mileage.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'negro', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'volkswagen', 'model': 'golf', 'trim': 'advance 1.6 tdi 110cv bmt', 'title': 'volkswagen golf advance 1.6 tdi 110cv bmt', 'price': 0, 'odometer': 409160, 'odometer_reading_unit': 'km', 'registration_date': '2016-10-19', 'year': 2016, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.6l', 'c_o_2_emission_value': 99, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '87da7f6e-0bb4-5ee7-8d3e-16de00674ea8', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_87da7f6e-0bb4-5ee7-8d3e-16de00674ea8/87da7f6e-0bb4-5ee7-8d3e-16de00674ea8_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/449/91a762a26b3842c5ac9e27ff8f9b6ebf/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 377019}], 'current_year_diff': 9, 'age_in_months': 109, 'make_model': 'volkswagen golf', 'sanitized_make': 'volkswagen', 'odometer_in_km': 409160}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 109,
     'auction_title': '',
     'c_o_2_emission_value': 99,
     'color': 'negro',
     'currency': 'eur',
     'current_year_diff': 9,
     'doors_number': 5,
     'engine': '1.6l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': '87da7f6e-0bb4-5ee7-8d3e-16de00674ea8',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
     'listing': 'auction',
     'location': None,
     'make': 'volkswagen',
     'make_model': 'volkswagen golf',
     'model': 'golf',
     'odometer': 409160,
     'odometer_in_km': 409160,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_87da7f6e-0bb4-5ee7-8d3e-16de00674ea8/87da7f6e-0bb4-5ee7-8d3e-16de00674ea8_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 377019,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/449/91a762a26b3842c5ac9e27ff8f9b6ebf/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2016-10-19',
     'sanitized_make': 'volkswagen',
     'seats_number': 5,
     'seller_name': None,
     'title': 'volkswagen golf advance 1.6 tdi 110cv bmt',
     'transmission': 'manual',
     'trim': 'advance 1.6 tdi 110cv bmt',
     'vin': None,
     'year': 2016}
{'auction_title': '',
 'c_o_2_emission_value': 99,
 'color': 'Negro',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.6L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '87da7f6e-0bb4-5ee7-8d3e-16de00674ea8',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'GOLF',
 'odometer': 409160,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_87da7f6e-0bb4-5ee7-8d3e-16de00674ea8/87da7f6e-0bb4-5ee7-8d3e-16de00674ea8_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 377019,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/449/91a762a26b3842c5ac9e27ff8f9b6ebf/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2016-10-19',
 'seats_number': 5,
 'seller_name': None,
 'title': 'VOLKSWAGEN GOLF Advance 1.6 TDI 110CV BMT',
 'transmission': 'manual',
 'trim': 'Advance 1.6 TDI 110CV BMT',
 'vin': None,
 'year': 2016}
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788887%27 HTTP/1.1" 200 None
2025-12-05 20:02:50 [ayvens.es] INFO: Saving data for 1788887: {'created_time': 1764964970.83862, 'last_price_update_time': 1764964970.83863, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788887') HTTP/1.1" 204 0
2025-12-05 20:02:50 [ayvens.es] INFO: Found listing with ID: 1788896
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788896%27 HTTP/1.1" 200 None
2025-12-05 20:02:50 [ayvens.es] INFO: Scrape type for 1788896: 1
2025-12-05 20:02:50 [ayvens.es] INFO: Extracting data for car ID: 1788896
2025-12-05 20:02:50 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 887d69c9-63b9-5a9d-8ca0-7328f211d56d with identifier: 1788896
2025-12-05 20:02:50 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 887d69c9-63b9-5a9d-8ca0-7328f211d56d with scrape type 1
2025-12-05 20:02:50 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 887d69c9-63b9-5a9d-8ca0-7328f211d56d
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F292%252F05e20d83addc4e689f9618452538985d%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:50 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/292/05e20d83addc4e689f9618452538985d/images/large/1.jpg not downloaded yet
2025-12-05 20:02:50 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/082/e3c1b10e6b7f4612bfc76b5858516ad5/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:50 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/068/9da01439420d44c6bb5669f33353cee4/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:50 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:50 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:50 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:50 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b05f9d10>
2025-12-05 20:02:50 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:50 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:51 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1969
2025-12-05 20:02:51 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:51 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:51 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:51 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_5cd1b8f7-8fb5-528c-afd0-1605c9143822/5cd1b8f7-8fb5-528c-afd0-1605c9143822_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F168%252F81e659889a5649809f06902517896947%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:51 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/168/81e659889a5649809f06902517896947/images/large/1.jpg saved to cache
2025-12-05 20:02:51 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 5cd1b8f7-8fb5-528c-afd0-1605c9143822
2025-12-05 20:02:51 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 5cd1b8f7-8fb5-528c-afd0-1605c9143822, skipping ID generation
2025-12-05 20:02:51 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 5cd1b8f7-8fb5-528c-afd0-1605c9143822 with scrape type 1
2025-12-05 20:02:51 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 5cd1b8f7-8fb5-528c-afd0-1605c9143822 from tracking dictionaries
2025-12-05 20:02:51 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 5cd1b8f7-8fb5-528c-afd0-1605c9143822 sending to next pipeline
2025-12-05 20:02:51 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 5cd1b8f7-8fb5-528c-afd0-1605c9143822, identifier: 1788889
2025-12-05 20:02:51 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:51 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:51 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:51 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b054ec50>
2025-12-05 20:02:51 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:51 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:51 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1971
2025-12-05 20:02:51 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:51 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:51 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:51 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_78d7b5ea-91eb-5003-9995-d33fc807f4ae/78d7b5ea-91eb-5003-9995-d33fc807f4ae_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F229%252F9ec3c358721d4291aeb2d1e127b5ca4d%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:51 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/229/9ec3c358721d4291aeb2d1e127b5ca4d/images/large/1.jpg saved to cache
2025-12-05 20:02:51 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 78d7b5ea-91eb-5003-9995-d33fc807f4ae
2025-12-05 20:02:51 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 78d7b5ea-91eb-5003-9995-d33fc807f4ae, skipping ID generation
2025-12-05 20:02:51 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 78d7b5ea-91eb-5003-9995-d33fc807f4ae with scrape type 1
2025-12-05 20:02:51 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 78d7b5ea-91eb-5003-9995-d33fc807f4ae from tracking dictionaries
2025-12-05 20:02:51 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 78d7b5ea-91eb-5003-9995-d33fc807f4ae sending to next pipeline
2025-12-05 20:02:51 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 78d7b5ea-91eb-5003-9995-d33fc807f4ae, identifier: 1788888
2025-12-05 20:02:51 [ayvens.es] INFO: Found listing with ID: 1788897
2025-12-05 20:02:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788897%27 HTTP/1.1" 200 None
2025-12-05 20:02:51 [ayvens.es] INFO: Scrape type for 1788897: 1
2025-12-05 20:02:51 [ayvens.es] INFO: Extracting data for car ID: 1788897
2025-12-05 20:02:51 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: b4e566ae-4c33-5cce-85a2-38f4e9060216 with identifier: 1788897
2025-12-05 20:02:51 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b4e566ae-4c33-5cce-85a2-38f4e9060216 with scrape type 1
2025-12-05 20:02:51 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item b4e566ae-4c33-5cce-85a2-38f4e9060216
2025-12-05 20:02:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F137%252F9a31c33fe0e74d1d91c1790951891f78%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:51 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/137/9a31c33fe0e74d1d91c1790951891f78/images/large/1.jpg not downloaded yet
2025-12-05 20:02:51 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 5cd1b8f7-8fb5-528c-afd0-1605c9143822 with identifier: 1788889
2025-12-05 20:02:51 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:02:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:02:51 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 5cd1b8f7-8fb5-528c-afd0-1605c9143822 with identifier: 1788889
2025-12-05 20:02:51 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 5cd1b8f7-8fb5-528c-afd0-1605c9143822 with identifier: 1788889 to the API
2025-12-05 20:02:51 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:52 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:52 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/168/81e659889a5649809f06902517896947/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 117,
 'color': 'Birch',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '5cd1b8f7-8fb5-528c-afd0-1605c9143822',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'VOLVO',
 'model': 'V60',
 'odometer': 81183,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_5cd1b8f7-8fb5-528c-afd0-1605c9143822/5cd1b8f7-8fb5-528c-afd0-1605c9143822_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 372243,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/168/81e659889a5649809f06902517896947/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-09-29',
 'seats_number': 5,
 'seller_name': None,
 'title': 'VOLVO V60 2.0 D3',
 'transmission': 'manual',
 'trim': '2.0 D3',
 'vin': None,
 'year': 2020}
2025-12-05 20:02:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788889%27 HTTP/1.1" 200 None
2025-12-05 20:02:52 [ayvens.es] INFO: Saving data for 1788889: {'created_time': 1764964972.192109, 'last_price_update_time': 1764964972.192122, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788889') HTTP/1.1" 204 0
2025-12-05 20:02:52 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 78d7b5ea-91eb-5003-9995-d33fc807f4ae with identifier: 1788888
2025-12-05 20:02:52 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 78d7b5ea-91eb-5003-9995-d33fc807f4ae with identifier: 1788888
2025-12-05 20:02:52 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 78d7b5ea-91eb-5003-9995-d33fc807f4ae with identifier: 1788888 to the API
2025-12-05 20:02:52 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:52 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:52 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/229/9ec3c358721d4291aeb2d1e127b5ca4d/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 142,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '78d7b5ea-91eb-5003-9995-d33fc807f4ae',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'AUDI',
 'model': 'Q3',
 'odometer': 103205,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_78d7b5ea-91eb-5003-9995-d33fc807f4ae/78d7b5ea-91eb-5003-9995-d33fc807f4ae_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 324497,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/229/9ec3c358721d4291aeb2d1e127b5ca4d/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-02-09',
 'seats_number': 5,
 'seller_name': None,
 'title': 'AUDI Q3 Advanced 35 TDI 110kW (150CV) S tronic',
 'transmission': 'manual',
 'trim': 'Advanced 35 TDI 110kW (150CV) S tronic',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788888%27 HTTP/1.1" 200 None
2025-12-05 20:02:52 [ayvens.es] INFO: Saving data for 1788888: {'created_time': 1764964972.560737, 'last_price_update_time': 1764964972.560748, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788888') HTTP/1.1" 204 0
2025-12-05 20:02:52 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/003/74b5a866d96f4062a2e419d1803fd148/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:52 [ayvens.es] INFO: Found listing with ID: 1788898
2025-12-05 20:02:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788898%27 HTTP/1.1" 200 None
2025-12-05 20:02:52 [ayvens.es] INFO: Scrape type for 1788898: 1
2025-12-05 20:02:52 [ayvens.es] INFO: Extracting data for car ID: 1788898
2025-12-05 20:02:52 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: e8451a95-704d-5716-bdc2-c858afa6804b with identifier: 1788898
2025-12-05 20:02:52 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e8451a95-704d-5716-bdc2-c858afa6804b with scrape type 1
2025-12-05 20:02:52 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item e8451a95-704d-5716-bdc2-c858afa6804b
2025-12-05 20:02:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F107%252Fb3721864b1844c2b9d4cf9b09f0cb8b6%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:52 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/107/b3721864b1844c2b9d4cf9b09f0cb8b6/images/large/1.jpg not downloaded yet
2025-12-05 20:02:52 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/846/8819adfe2fd54957ba4fc4c15f2d490b/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:52 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:52 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:52 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:52 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b0d810>
2025-12-05 20:02:52 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:52 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:52 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:02:52 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:52 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:52 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:52 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_c8b92838-cc17-5c2f-acb6-1b649dcab627/c8b92838-cc17-5c2f-acb6-1b649dcab627_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F082%252Fe3c1b10e6b7f4612bfc76b5858516ad5%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:52 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/082/e3c1b10e6b7f4612bfc76b5858516ad5/images/large/1.jpg saved to cache
2025-12-05 20:02:52 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item c8b92838-cc17-5c2f-acb6-1b649dcab627
2025-12-05 20:02:52 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: c8b92838-cc17-5c2f-acb6-1b649dcab627, skipping ID generation
2025-12-05 20:02:52 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item c8b92838-cc17-5c2f-acb6-1b649dcab627 with scrape type 1
2025-12-05 20:02:52 [crawlers.middlewares.photo_download_middleware] INFO: Removing item c8b92838-cc17-5c2f-acb6-1b649dcab627 from tracking dictionaries
2025-12-05 20:02:52 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item c8b92838-cc17-5c2f-acb6-1b649dcab627 sending to next pipeline
2025-12-05 20:02:53 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: c8b92838-cc17-5c2f-acb6-1b649dcab627, identifier: 1788890
2025-12-05 20:02:53 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:53 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:53 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:53 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0403810>
2025-12-05 20:02:53 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:53 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:53 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:02:53 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:53 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:53 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:53 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_e01808eb-4d99-5c8d-8386-5c90a3b8cbbd/e01808eb-4d99-5c8d-8386-5c90a3b8cbbd_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F068%252F9da01439420d44c6bb5669f33353cee4%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:53 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/068/9da01439420d44c6bb5669f33353cee4/images/large/1.jpg saved to cache
2025-12-05 20:02:53 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item e01808eb-4d99-5c8d-8386-5c90a3b8cbbd
2025-12-05 20:02:53 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: e01808eb-4d99-5c8d-8386-5c90a3b8cbbd, skipping ID generation
2025-12-05 20:02:53 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e01808eb-4d99-5c8d-8386-5c90a3b8cbbd with scrape type 1
2025-12-05 20:02:53 [crawlers.middlewares.photo_download_middleware] INFO: Removing item e01808eb-4d99-5c8d-8386-5c90a3b8cbbd from tracking dictionaries
2025-12-05 20:02:53 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item e01808eb-4d99-5c8d-8386-5c90a3b8cbbd sending to next pipeline
2025-12-05 20:02:53 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: e01808eb-4d99-5c8d-8386-5c90a3b8cbbd, identifier: 1788891
2025-12-05 20:02:53 [ayvens.es] INFO: Found listing with ID: 1788899
2025-12-05 20:02:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788899%27 HTTP/1.1" 200 None
2025-12-05 20:02:53 [ayvens.es] INFO: Scrape type for 1788899: 1
2025-12-05 20:02:53 [ayvens.es] INFO: Extracting data for car ID: 1788899
2025-12-05 20:02:53 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 70e69dfd-c1d4-5031-bf2e-3a17403f765b with identifier: 1788899
2025-12-05 20:02:53 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 70e69dfd-c1d4-5031-bf2e-3a17403f765b with scrape type 1
2025-12-05 20:02:53 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 70e69dfd-c1d4-5031-bf2e-3a17403f765b
2025-12-05 20:02:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F644%252F808a65dcc3c8405088934dff0be60a20%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:53 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/644/808a65dcc3c8405088934dff0be60a20/images/large/1.jpg not downloaded yet
2025-12-05 20:02:53 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: c8b92838-cc17-5c2f-acb6-1b649dcab627 with identifier: 1788890
2025-12-05 20:02:53 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: c8b92838-cc17-5c2f-acb6-1b649dcab627 with identifier: 1788890
2025-12-05 20:02:53 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: c8b92838-cc17-5c2f-acb6-1b649dcab627 with identifier: 1788890 to the API
2025-12-05 20:02:53 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:53 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:53 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/082/e3c1b10e6b7f4612bfc76b5858516ad5/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 130,
 'color': 'Negro',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'c8b92838-cc17-5c2f-acb6-1b649dcab627',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SEAT',
 'model': 'ATECA',
 'odometer': 111726,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_c8b92838-cc17-5c2f-acb6-1b649dcab627/c8b92838-cc17-5c2f-acb6-1b649dcab627_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 378294,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/082/e3c1b10e6b7f4612bfc76b5858516ad5/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-10-22',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SEAT ATECA 2.0 TDI 110kW (150CV) DSG S&S Style Go',
 'transmission': 'manual',
 'trim': '2.0 TDI 110kW (150CV) DSG S&S Style Go',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788890%27 HTTP/1.1" 200 None
2025-12-05 20:02:53 [ayvens.es] INFO: Saving data for 1788890: {'created_time': 1764964973.997587, 'last_price_update_time': 1764964973.9976, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788890') HTTP/1.1" 204 0
2025-12-05 20:02:54 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: e01808eb-4d99-5c8d-8386-5c90a3b8cbbd with identifier: 1788891
2025-12-05 20:02:54 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: e01808eb-4d99-5c8d-8386-5c90a3b8cbbd with identifier: 1788891
2025-12-05 20:02:54 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: e01808eb-4d99-5c8d-8386-5c90a3b8cbbd with identifier: 1788891 to the API
2025-12-05 20:02:54 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:54 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:54 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/068/9da01439420d44c6bb5669f33353cee4/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 102,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.8L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'hybrid',
 'id': 'e01808eb-4d99-5c8d-8386-5c90a3b8cbbd',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'TOYOTA',
 'model': 'COROLLA',
 'odometer': 41878,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_e01808eb-4d99-5c8d-8386-5c90a3b8cbbd/e01808eb-4d99-5c8d-8386-5c90a3b8cbbd_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 339308,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/068/9da01439420d44c6bb5669f33353cee4/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2022-05-09',
 'seats_number': 5,
 'seller_name': None,
 'title': 'TOYOTA COROLLA 1.8 125H ACTIVE TECH E-CVT',
 'transmission': 'manual',
 'trim': '1.8 125H ACTIVE TECH E-CVT',
 'vin': None,
 'year': 2022}
2025-12-05 20:02:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788891%27 HTTP/1.1" 200 None
2025-12-05 20:02:54 [ayvens.es] INFO: Saving data for 1788891: {'created_time': 1764964974.37919, 'last_price_update_time': 1764964974.3792, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788891') HTTP/1.1" 204 0
2025-12-05 20:02:54 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/343/31f78186538c486daeccc46697cd1a0c/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:54 [ayvens.es] INFO: Found listing with ID: 1788900
2025-12-05 20:02:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788900%27 HTTP/1.1" 200 None
2025-12-05 20:02:54 [ayvens.es] INFO: Scrape type for 1788900: 1
2025-12-05 20:02:54 [ayvens.es] INFO: Extracting data for car ID: 1788900
2025-12-05 20:02:54 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: e29098f8-163d-57fc-a241-ece190617f9c with identifier: 1788900
2025-12-05 20:02:54 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e29098f8-163d-57fc-a241-ece190617f9c with scrape type 1
2025-12-05 20:02:54 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item e29098f8-163d-57fc-a241-ece190617f9c
2025-12-05 20:02:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F280%252Fda713551a2b34e23b198e7a29daa9bdb%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:54 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/280/da713551a2b34e23b198e7a29daa9bdb/images/large/1.jpg not downloaded yet
2025-12-05 20:02:54 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/805/2d8594d13f9c4439b78745db43f691ac/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:54 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:54 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:54 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:54 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0ab9990>
2025-12-05 20:02:54 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:54 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:54 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:02:54 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:54 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:54 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:54 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_7d85304b-0ea0-5784-a1ee-bb0fdb7cde36/7d85304b-0ea0-5784-a1ee-bb0fdb7cde36_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F003%252F74b5a866d96f4062a2e419d1803fd148%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:54 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/003/74b5a866d96f4062a2e419d1803fd148/images/large/1.jpg saved to cache
2025-12-05 20:02:54 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 7d85304b-0ea0-5784-a1ee-bb0fdb7cde36
2025-12-05 20:02:54 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 7d85304b-0ea0-5784-a1ee-bb0fdb7cde36, skipping ID generation
2025-12-05 20:02:54 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 7d85304b-0ea0-5784-a1ee-bb0fdb7cde36 with scrape type 1
2025-12-05 20:02:54 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 7d85304b-0ea0-5784-a1ee-bb0fdb7cde36 from tracking dictionaries
2025-12-05 20:02:54 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 7d85304b-0ea0-5784-a1ee-bb0fdb7cde36 sending to next pipeline
2025-12-05 20:02:54 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 7d85304b-0ea0-5784-a1ee-bb0fdb7cde36, identifier: 1788892
2025-12-05 20:02:54 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:54 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:54 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:54 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b28ecc90>
2025-12-05 20:02:54 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:54 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:55 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:02:55 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:55 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:55 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:55 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_824142ed-f7e4-512e-b5ef-32d753cf3a9a/824142ed-f7e4-512e-b5ef-32d753cf3a9a_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F846%252F8819adfe2fd54957ba4fc4c15f2d490b%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:55 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/846/8819adfe2fd54957ba4fc4c15f2d490b/images/large/1.jpg saved to cache
2025-12-05 20:02:55 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 824142ed-f7e4-512e-b5ef-32d753cf3a9a
2025-12-05 20:02:55 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 824142ed-f7e4-512e-b5ef-32d753cf3a9a, skipping ID generation
2025-12-05 20:02:55 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 824142ed-f7e4-512e-b5ef-32d753cf3a9a with scrape type 1
2025-12-05 20:02:55 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 824142ed-f7e4-512e-b5ef-32d753cf3a9a from tracking dictionaries
2025-12-05 20:02:55 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 824142ed-f7e4-512e-b5ef-32d753cf3a9a sending to next pipeline
2025-12-05 20:02:55 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 824142ed-f7e4-512e-b5ef-32d753cf3a9a, identifier: 1788893
2025-12-05 20:02:55 [ayvens.es] INFO: Found listing with ID: 1788901
2025-12-05 20:02:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788901%27 HTTP/1.1" 200 None
2025-12-05 20:02:55 [ayvens.es] INFO: Scrape type for 1788901: 1
2025-12-05 20:02:55 [ayvens.es] INFO: Extracting data for car ID: 1788901
2025-12-05 20:02:55 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: f1a13805-f134-5a1d-82ee-434a1c8ce9a0 with identifier: 1788901
2025-12-05 20:02:55 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f1a13805-f134-5a1d-82ee-434a1c8ce9a0 with scrape type 1
2025-12-05 20:02:55 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item f1a13805-f134-5a1d-82ee-434a1c8ce9a0
2025-12-05 20:02:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F275%252Fbd636151ee6446609e38e6e3213fb46d%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:55 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/275/bd636151ee6446609e38e6e3213fb46d/images/large/1.jpg not downloaded yet
2025-12-05 20:02:55 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 7d85304b-0ea0-5784-a1ee-bb0fdb7cde36 with identifier: 1788892
2025-12-05 20:02:55 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 7d85304b-0ea0-5784-a1ee-bb0fdb7cde36 with identifier: 1788892
2025-12-05 20:02:55 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 7d85304b-0ea0-5784-a1ee-bb0fdb7cde36 with identifier: 1788892 to the API
2025-12-05 20:02:55 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:55 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:55 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/003/74b5a866d96f4062a2e419d1803fd148/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 130,
 'color': 'Saphirschwarz',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '7d85304b-0ea0-5784-a1ee-bb0fdb7cde36',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'BMW',
 'model': 'X1',
 'odometer': 111730,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_7d85304b-0ea0-5784-a1ee-bb0fdb7cde36/7d85304b-0ea0-5784-a1ee-bb0fdb7cde36_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 360787,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/003/74b5a866d96f4062a2e419d1803fd148/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-07-21',
 'seats_number': 5,
 'seller_name': None,
 'title': 'BMW X1 sDrive18d',
 'transmission': 'manual',
 'trim': 'sDrive18d',
 'vin': None,
 'year': 2021}
2025-12-05 20:02:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788892%27 HTTP/1.1" 200 None
2025-12-05 20:02:55 [ayvens.es] INFO: Saving data for 1788892: {'created_time': 1764964975.692267, 'last_price_update_time': 1764964975.692278, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788892') HTTP/1.1" 204 0
2025-12-05 20:02:55 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 824142ed-f7e4-512e-b5ef-32d753cf3a9a with identifier: 1788893
2025-12-05 20:02:55 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 824142ed-f7e4-512e-b5ef-32d753cf3a9a with identifier: 1788893
2025-12-05 20:02:55 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 824142ed-f7e4-512e-b5ef-32d753cf3a9a with identifier: 1788893 to the API
2025-12-05 20:02:55 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:57 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:57 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/846/8819adfe2fd54957ba4fc4c15f2d490b/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 152,
 'color': 'Negro',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '824142ed-f7e4-512e-b5ef-32d753cf3a9a',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'CUPRA',
 'model': 'FORMENTOR',
 'odometer': 24987,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_824142ed-f7e4-512e-b5ef-32d753cf3a9a/824142ed-f7e4-512e-b5ef-32d753cf3a9a_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 362793,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/846/8819adfe2fd54957ba4fc4c15f2d490b/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2024-09-20',
 'seats_number': 5,
 'seller_name': None,
 'title': 'CUPRA FORMENTOR 1.5 TSI 110kW (150 CV) DSG',
 'transmission': 'manual',
 'trim': '1.5 TSI 110kW (150 CV) DSG',
 'vin': None,
 'year': 2024}
2025-12-05 20:02:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788893%27 HTTP/1.1" 200 None
2025-12-05 20:02:57 [ayvens.es] INFO: Saving data for 1788893: {'created_time': 1764964977.706127, 'last_price_update_time': 1764964977.706138, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788893') HTTP/1.1" 204 0
2025-12-05 20:02:57 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/292/05e20d83addc4e689f9618452538985d/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:57 [ayvens.es] INFO: Found listing with ID: 1788902
2025-12-05 20:02:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788902%27 HTTP/1.1" 200 None
2025-12-05 20:02:57 [ayvens.es] INFO: Scrape type for 1788902: 1
2025-12-05 20:02:57 [ayvens.es] INFO: Extracting data for car ID: 1788902
2025-12-05 20:02:57 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 9fac1d64-584c-5fa9-928d-807fdf792dd2 with identifier: 1788902
2025-12-05 20:02:57 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9fac1d64-584c-5fa9-928d-807fdf792dd2 with scrape type 1
2025-12-05 20:02:57 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 9fac1d64-584c-5fa9-928d-807fdf792dd2
2025-12-05 20:02:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F528%252F94a119b1869340e196661ad2c7fb3e16%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:57 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/528/94a119b1869340e196661ad2c7fb3e16/images/large/1.jpg not downloaded yet
2025-12-05 20:02:57 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/137/9a31c33fe0e74d1d91c1790951891f78/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:57 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:57 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:57 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:57 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b06b3a50>
2025-12-05 20:02:57 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:57 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:58 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:02:58 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:58 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:58 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:58 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_04137861-8d1b-5ef9-a869-87207ac8e377/04137861-8d1b-5ef9-a869-87207ac8e377_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F343%252F31f78186538c486daeccc46697cd1a0c%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:58 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/343/31f78186538c486daeccc46697cd1a0c/images/large/1.jpg saved to cache
2025-12-05 20:02:58 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 04137861-8d1b-5ef9-a869-87207ac8e377
2025-12-05 20:02:58 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 04137861-8d1b-5ef9-a869-87207ac8e377, skipping ID generation
2025-12-05 20:02:58 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 04137861-8d1b-5ef9-a869-87207ac8e377 with scrape type 1
2025-12-05 20:02:58 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 04137861-8d1b-5ef9-a869-87207ac8e377 from tracking dictionaries
2025-12-05 20:02:58 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 04137861-8d1b-5ef9-a869-87207ac8e377 sending to next pipeline
2025-12-05 20:02:58 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 04137861-8d1b-5ef9-a869-87207ac8e377, identifier: 1788894
2025-12-05 20:02:58 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:58 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:58 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:58 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0d5e210>
2025-12-05 20:02:58 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:58 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:58 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:02:58 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:58 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:58 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:02:58 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_44579e26-a161-50b0-b1e1-d2efe0176205/44579e26-a161-50b0-b1e1-d2efe0176205_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:02:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F805%252F2d8594d13f9c4439b78745db43f691ac%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:02:58 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/805/2d8594d13f9c4439b78745db43f691ac/images/large/1.jpg saved to cache
2025-12-05 20:02:58 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 44579e26-a161-50b0-b1e1-d2efe0176205
2025-12-05 20:02:58 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 44579e26-a161-50b0-b1e1-d2efe0176205, skipping ID generation
2025-12-05 20:02:58 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 44579e26-a161-50b0-b1e1-d2efe0176205 with scrape type 1
2025-12-05 20:02:58 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 44579e26-a161-50b0-b1e1-d2efe0176205 from tracking dictionaries
2025-12-05 20:02:58 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 44579e26-a161-50b0-b1e1-d2efe0176205 sending to next pipeline
2025-12-05 20:02:58 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 44579e26-a161-50b0-b1e1-d2efe0176205, identifier: 1788895
2025-12-05 20:02:58 [ayvens.es] INFO: Found listing with ID: 1788903
2025-12-05 20:02:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788903%27 HTTP/1.1" 200 None
2025-12-05 20:02:58 [ayvens.es] INFO: Scrape type for 1788903: 1
2025-12-05 20:02:58 [ayvens.es] INFO: Extracting data for car ID: 1788903
2025-12-05 20:02:58 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: e9f87121-c76a-5536-bd4a-4072ebb11155 with identifier: 1788903
2025-12-05 20:02:58 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e9f87121-c76a-5536-bd4a-4072ebb11155 with scrape type 1
2025-12-05 20:02:58 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item e9f87121-c76a-5536-bd4a-4072ebb11155
2025-12-05 20:02:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F352%252F966c19b0779e4ac682a423bca0e2d799%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:58 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/352/966c19b0779e4ac682a423bca0e2d799/images/large/1.jpg not downloaded yet
2025-12-05 20:02:58 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 04137861-8d1b-5ef9-a869-87207ac8e377 with identifier: 1788894
2025-12-05 20:02:58 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 04137861-8d1b-5ef9-a869-87207ac8e377 with identifier: 1788894
2025-12-05 20:02:58 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'toyota' containing 'toyota' and model 'hilux' containing 'hilux'
2025-12-05 20:02:58 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 04137861-8d1b-5ef9-a869-87207ac8e377 spider: ayvens.es
2025-12-05 20:02:59 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 04137861-8d1b-5ef9-a869-87207ac8e377 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'toyota' containing 'toyota' and model 'hilux' containing 'hilux'
{'auction_title': '',
 'c_o_2_emission_value': 187,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 2,
 'engine': '2.4L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '04137861-8d1b-5ef9-a869-87207ac8e377',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'TOYOTA',
 'model': 'HILUX',
 'odometer': 69806,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_04137861-8d1b-5ef9-a869-87207ac8e377/04137861-8d1b-5ef9-a869-87207ac8e377_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 341723,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/343/31f78186538c486daeccc46697cd1a0c/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-05-06',
 'seats_number': 2,
 'seller_name': None,
 'title': 'TOYOTA HILUX 2.4 D-4D Cabina Sencilla GX',
 'transmission': 'manual',
 'trim': '2.4 D-4D Cabina Sencilla GX',
 'vin': None,
 'year': 2019}
2025-12-05 20:02:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788894%27 HTTP/1.1" 200 None
2025-12-05 20:02:59 [ayvens.es] INFO: Saving data for 1788894: {'created_time': 1764964979.054183, 'last_price_update_time': 1764964979.054193, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788894') HTTP/1.1" 204 0
2025-12-05 20:02:59 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 44579e26-a161-50b0-b1e1-d2efe0176205 with identifier: 1788895
2025-12-05 20:02:59 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 44579e26-a161-50b0-b1e1-d2efe0176205 with identifier: 1788895
2025-12-05 20:02:59 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 44579e26-a161-50b0-b1e1-d2efe0176205 with identifier: 1788895 to the API
2025-12-05 20:02:59 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:02:59 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:02:59 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/805/2d8594d13f9c4439b78745db43f691ac/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 76,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.8L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'hybrid',
 'id': '44579e26-a161-50b0-b1e1-d2efe0176205',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'TOYOTA',
 'model': 'COROLLA',
 'odometer': 22334,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_44579e26-a161-50b0-b1e1-d2efe0176205/44579e26-a161-50b0-b1e1-d2efe0176205_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 337973,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/805/2d8594d13f9c4439b78745db43f691ac/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-02-12',
 'seats_number': 5,
 'seller_name': None,
 'title': 'TOYOTA COROLLA 1.8 125H ACTIVE E-CVT',
 'transmission': 'manual',
 'trim': '1.8 125H ACTIVE E-CVT',
 'vin': None,
 'year': 2020}
2025-12-05 20:02:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788895%27 HTTP/1.1" 200 None
2025-12-05 20:02:59 [ayvens.es] INFO: Saving data for 1788895: {'created_time': 1764964979.446296, 'last_price_update_time': 1764964979.446308, 'auction_closing_time': 1765281600.0}
2025-12-05 20:02:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788895') HTTP/1.1" 204 0
2025-12-05 20:02:59 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/107/b3721864b1844c2b9d4cf9b09f0cb8b6/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:59 [ayvens.es] INFO: Found listing with ID: 1788904
2025-12-05 20:02:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788904%27 HTTP/1.1" 200 None
2025-12-05 20:02:59 [ayvens.es] INFO: Scrape type for 1788904: 1
2025-12-05 20:02:59 [ayvens.es] INFO: Extracting data for car ID: 1788904
2025-12-05 20:02:59 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 9eec7ff8-1586-5785-ac80-1ec043fc4596 with identifier: 1788904
2025-12-05 20:02:59 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9eec7ff8-1586-5785-ac80-1ec043fc4596 with scrape type 1
2025-12-05 20:02:59 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 9eec7ff8-1586-5785-ac80-1ec043fc4596
2025-12-05 20:02:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:02:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F672%252F414c295b73d842b995ad8d13d5cb7c35%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:02:59 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/672/414c295b73d842b995ad8d13d5cb7c35/images/large/1.jpg not downloaded yet
2025-12-05 20:02:59 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/644/808a65dcc3c8405088934dff0be60a20/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:02:59 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:02:59 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:02:59 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:02:59 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0cf8390>
2025-12-05 20:02:59 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:02:59 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:02:59 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:02:59 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:02:59 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:02:59 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:00 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_887d69c9-63b9-5a9d-8ca0-7328f211d56d/887d69c9-63b9-5a9d-8ca0-7328f211d56d_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F292%252F05e20d83addc4e689f9618452538985d%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:00 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/292/05e20d83addc4e689f9618452538985d/images/large/1.jpg saved to cache
2025-12-05 20:03:00 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 887d69c9-63b9-5a9d-8ca0-7328f211d56d
2025-12-05 20:03:00 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 887d69c9-63b9-5a9d-8ca0-7328f211d56d, skipping ID generation
2025-12-05 20:03:00 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 887d69c9-63b9-5a9d-8ca0-7328f211d56d with scrape type 1
2025-12-05 20:03:00 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 887d69c9-63b9-5a9d-8ca0-7328f211d56d from tracking dictionaries
2025-12-05 20:03:00 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 887d69c9-63b9-5a9d-8ca0-7328f211d56d sending to next pipeline
2025-12-05 20:03:00 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 887d69c9-63b9-5a9d-8ca0-7328f211d56d, identifier: 1788896
2025-12-05 20:03:00 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:00 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:00 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:00 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0ad9850>
2025-12-05 20:03:00 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:00 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:00 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:03:00 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:00 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:00 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:00 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_b4e566ae-4c33-5cce-85a2-38f4e9060216/b4e566ae-4c33-5cce-85a2-38f4e9060216_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F137%252F9a31c33fe0e74d1d91c1790951891f78%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:00 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/137/9a31c33fe0e74d1d91c1790951891f78/images/large/1.jpg saved to cache
2025-12-05 20:03:00 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item b4e566ae-4c33-5cce-85a2-38f4e9060216
2025-12-05 20:03:00 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: b4e566ae-4c33-5cce-85a2-38f4e9060216, skipping ID generation
2025-12-05 20:03:00 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b4e566ae-4c33-5cce-85a2-38f4e9060216 with scrape type 1
2025-12-05 20:03:00 [crawlers.middlewares.photo_download_middleware] INFO: Removing item b4e566ae-4c33-5cce-85a2-38f4e9060216 from tracking dictionaries
2025-12-05 20:03:00 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item b4e566ae-4c33-5cce-85a2-38f4e9060216 sending to next pipeline
2025-12-05 20:03:00 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: b4e566ae-4c33-5cce-85a2-38f4e9060216, identifier: 1788897
2025-12-05 20:03:00 [ayvens.es] INFO: Found listing with ID: 1788905
2025-12-05 20:03:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788905%27 HTTP/1.1" 200 None
2025-12-05 20:03:00 [ayvens.es] INFO: Scrape type for 1788905: 1
2025-12-05 20:03:00 [ayvens.es] INFO: Extracting data for car ID: 1788905
2025-12-05 20:03:00 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 411b2b41-33e5-5062-863e-a9eabde3e8e9 with identifier: 1788905
2025-12-05 20:03:00 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 411b2b41-33e5-5062-863e-a9eabde3e8e9 with scrape type 1
2025-12-05 20:03:00 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 411b2b41-33e5-5062-863e-a9eabde3e8e9
2025-12-05 20:03:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F725%252Fc1c0d3db26eb4bcb8341e192fbcc1ebe%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:00 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/725/c1c0d3db26eb4bcb8341e192fbcc1ebe/images/large/1.jpg not downloaded yet
2025-12-05 20:03:00 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 887d69c9-63b9-5a9d-8ca0-7328f211d56d with identifier: 1788896
2025-12-05 20:03:00 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 887d69c9-63b9-5a9d-8ca0-7328f211d56d with identifier: 1788896
2025-12-05 20:03:00 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 887d69c9-63b9-5a9d-8ca0-7328f211d56d with identifier: 1788896 to the API
2025-12-05 20:03:00 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:00 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:01 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/292/05e20d83addc4e689f9618452538985d/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 120,
 'color': 'Negro',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '887d69c9-63b9-5a9d-8ca0-7328f211d56d',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': '508',
 'odometer': 31747,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_887d69c9-63b9-5a9d-8ca0-7328f211d56d/887d69c9-63b9-5a9d-8ca0-7328f211d56d_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 369003,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/292/05e20d83addc4e689f9618452538985d/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2022-09-22',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT 508 5P Active Pack BlueHDi 130 S&S EAT8',
 'transmission': 'manual',
 'trim': '5P Active Pack BlueHDi 130 S&S EAT8',
 'vin': None,
 'year': 2022}
2025-12-05 20:03:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788896%27 HTTP/1.1" 200 None
2025-12-05 20:03:01 [ayvens.es] INFO: Saving data for 1788896: {'created_time': 1764964981.069446, 'last_price_update_time': 1764964981.069455, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788896') HTTP/1.1" 204 0
2025-12-05 20:03:01 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: b4e566ae-4c33-5cce-85a2-38f4e9060216 with identifier: 1788897
2025-12-05 20:03:01 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: b4e566ae-4c33-5cce-85a2-38f4e9060216 with identifier: 1788897
2025-12-05 20:03:01 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: b4e566ae-4c33-5cce-85a2-38f4e9060216 with identifier: 1788897 to the API
2025-12-05 20:03:01 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:01 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:01 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/137/9a31c33fe0e74d1d91c1790951891f78/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 150,
 'color': 'Azul',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.6L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': 'b4e566ae-4c33-5cce-85a2-38f4e9060216',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'KIA',
 'model': 'CARENS',
 'odometer': 27581,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_b4e566ae-4c33-5cce-85a2-38f4e9060216/b4e566ae-4c33-5cce-85a2-38f4e9060216_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 362490,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/137/9a31c33fe0e74d1d91c1790951891f78/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2018-10-15',
 'seats_number': 5,
 'seller_name': None,
 'title': 'KIA CARENS 1.6 GDi 99kW (135CV) Concept',
 'transmission': 'manual',
 'trim': '1.6 GDi 99kW (135CV) Concept',
 'vin': None,
 'year': 2018}
2025-12-05 20:03:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788897%27 HTTP/1.1" 200 None
2025-12-05 20:03:01 [ayvens.es] INFO: Saving data for 1788897: {'created_time': 1764964981.504162, 'last_price_update_time': 1764964981.50417, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788897') HTTP/1.1" 204 0
2025-12-05 20:03:01 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/280/da713551a2b34e23b198e7a29daa9bdb/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:01 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/275/bd636151ee6446609e38e6e3213fb46d/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:01 [ayvens.es] INFO: Found listing with ID: 1788906
2025-12-05 20:03:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788906%27 HTTP/1.1" 200 None
2025-12-05 20:03:01 [ayvens.es] INFO: Scrape type for 1788906: 1
2025-12-05 20:03:01 [ayvens.es] INFO: Extracting data for car ID: 1788906
2025-12-05 20:03:01 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: f32ffefe-4ebb-58e3-942b-e5a2681ac5e3 with identifier: 1788906
2025-12-05 20:03:01 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f32ffefe-4ebb-58e3-942b-e5a2681ac5e3 with scrape type 1
2025-12-05 20:03:01 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item f32ffefe-4ebb-58e3-942b-e5a2681ac5e3
2025-12-05 20:03:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F806%252F6f8afbbf23554fa385b64cf5e11c2dca%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:01 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/806/6f8afbbf23554fa385b64cf5e11c2dca/images/large/1.jpg not downloaded yet
2025-12-05 20:03:01 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:01 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:01 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:01 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a40610>
2025-12-05 20:03:01 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:01 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:01 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:03:01 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:01 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:01 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:01 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_e8451a95-704d-5716-bdc2-c858afa6804b/e8451a95-704d-5716-bdc2-c858afa6804b_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F107%252Fb3721864b1844c2b9d4cf9b09f0cb8b6%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:01 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/107/b3721864b1844c2b9d4cf9b09f0cb8b6/images/large/1.jpg saved to cache
2025-12-05 20:03:01 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item e8451a95-704d-5716-bdc2-c858afa6804b
2025-12-05 20:03:01 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: e8451a95-704d-5716-bdc2-c858afa6804b, skipping ID generation
2025-12-05 20:03:01 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e8451a95-704d-5716-bdc2-c858afa6804b with scrape type 1
2025-12-05 20:03:01 [crawlers.middlewares.photo_download_middleware] INFO: Removing item e8451a95-704d-5716-bdc2-c858afa6804b from tracking dictionaries
2025-12-05 20:03:01 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item e8451a95-704d-5716-bdc2-c858afa6804b sending to next pipeline
2025-12-05 20:03:02 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: e8451a95-704d-5716-bdc2-c858afa6804b, identifier: 1788898
2025-12-05 20:03:02 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:02 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:02 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:02 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b71e50>
2025-12-05 20:03:02 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:02 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:02 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:03:02 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:02 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:02 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:02 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_70e69dfd-c1d4-5031-bf2e-3a17403f765b/70e69dfd-c1d4-5031-bf2e-3a17403f765b_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F644%252F808a65dcc3c8405088934dff0be60a20%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:02 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/644/808a65dcc3c8405088934dff0be60a20/images/large/1.jpg saved to cache
2025-12-05 20:03:02 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 70e69dfd-c1d4-5031-bf2e-3a17403f765b
2025-12-05 20:03:02 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 70e69dfd-c1d4-5031-bf2e-3a17403f765b, skipping ID generation
2025-12-05 20:03:02 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 70e69dfd-c1d4-5031-bf2e-3a17403f765b with scrape type 1
2025-12-05 20:03:02 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 70e69dfd-c1d4-5031-bf2e-3a17403f765b from tracking dictionaries
2025-12-05 20:03:02 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 70e69dfd-c1d4-5031-bf2e-3a17403f765b sending to next pipeline
2025-12-05 20:03:02 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 70e69dfd-c1d4-5031-bf2e-3a17403f765b, identifier: 1788899
2025-12-05 20:03:02 [ayvens.es] INFO: Found listing with ID: 1788907
2025-12-05 20:03:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788907%27 HTTP/1.1" 200 None
2025-12-05 20:03:02 [ayvens.es] INFO: Scrape type for 1788907: 1
2025-12-05 20:03:02 [ayvens.es] INFO: Extracting data for car ID: 1788907
2025-12-05 20:03:02 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 63002157-1c41-58c1-9117-b75b5574d5be with identifier: 1788907
2025-12-05 20:03:02 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 63002157-1c41-58c1-9117-b75b5574d5be with scrape type 1
2025-12-05 20:03:02 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 63002157-1c41-58c1-9117-b75b5574d5be
2025-12-05 20:03:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F594%252F2ea163777fca41ccba428d8a468bb363%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:02 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/594/2ea163777fca41ccba428d8a468bb363/images/large/1.jpg not downloaded yet
2025-12-05 20:03:02 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: e8451a95-704d-5716-bdc2-c858afa6804b with identifier: 1788898
2025-12-05 20:03:02 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: e8451a95-704d-5716-bdc2-c858afa6804b with identifier: 1788898
2025-12-05 20:03:02 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: e8451a95-704d-5716-bdc2-c858afa6804b with identifier: 1788898 to the API
2025-12-05 20:03:02 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:02 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:02 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/107/b3721864b1844c2b9d4cf9b09f0cb8b6/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 119,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'e8451a95-704d-5716-bdc2-c858afa6804b',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'GOLF',
 'odometer': 119006,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_e8451a95-704d-5716-bdc2-c858afa6804b/e8451a95-704d-5716-bdc2-c858afa6804b_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 330954,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/107/b3721864b1844c2b9d4cf9b09f0cb8b6/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-09-06',
 'seats_number': 5,
 'seller_name': None,
 'title': 'VOLKSWAGEN GOLF Life 2.0 TDI 85kW (115CV)',
 'transmission': 'manual',
 'trim': 'Life 2.0 TDI 85kW (115CV)',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788898%27 HTTP/1.1" 200 None
2025-12-05 20:03:02 [ayvens.es] INFO: Saving data for 1788898: {'created_time': 1764964982.824583, 'last_price_update_time': 1764964982.824595, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788898') HTTP/1.1" 204 0
2025-12-05 20:03:02 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 70e69dfd-c1d4-5031-bf2e-3a17403f765b with identifier: 1788899
2025-12-05 20:03:02 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 70e69dfd-c1d4-5031-bf2e-3a17403f765b with identifier: 1788899
2025-12-05 20:03:02 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 70e69dfd-c1d4-5031-bf2e-3a17403f765b with identifier: 1788899 to the API
2025-12-05 20:03:02 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:03 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:03 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/644/808a65dcc3c8405088934dff0be60a20/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 156,
 'color': 'Rojo',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '70e69dfd-c1d4-5031-bf2e-3a17403f765b',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SEAT',
 'model': 'ATECA',
 'odometer': 84038,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_70e69dfd-c1d4-5031-bf2e-3a17403f765b/70e69dfd-c1d4-5031-bf2e-3a17403f765b_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 376037,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/644/808a65dcc3c8405088934dff0be60a20/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-08-12',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SEAT ATECA 2.0 TSI 140kW DSG-7 4D S&S FR Edition',
 'transmission': 'manual',
 'trim': '2.0 TSI 140kW DSG-7 4D S&S FR Edition',
 'vin': None,
 'year': 2020}
2025-12-05 20:03:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788899%27 HTTP/1.1" 200 None
2025-12-05 20:03:03 [ayvens.es] INFO: Saving data for 1788899: {'created_time': 1764964983.193181, 'last_price_update_time': 1764964983.193193, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788899') HTTP/1.1" 204 0
2025-12-05 20:03:03 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/528/94a119b1869340e196661ad2c7fb3e16/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:03 [ayvens.es] INFO: Found listing with ID: 1788908
2025-12-05 20:03:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788908%27 HTTP/1.1" 200 None
2025-12-05 20:03:03 [ayvens.es] INFO: Scrape type for 1788908: 1
2025-12-05 20:03:03 [ayvens.es] INFO: Extracting data for car ID: 1788908
2025-12-05 20:03:03 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 15326f39-ea91-5b1c-8f72-8155a26e072f with identifier: 1788908
2025-12-05 20:03:03 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 15326f39-ea91-5b1c-8f72-8155a26e072f with scrape type 1
2025-12-05 20:03:03 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 15326f39-ea91-5b1c-8f72-8155a26e072f
2025-12-05 20:03:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F534%252Ffd7541a47881435b8cf3f6ece86e0247%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:03 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/534/fd7541a47881435b8cf3f6ece86e0247/images/large/1.jpg not downloaded yet
2025-12-05 20:03:03 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:03 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:03 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:03 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b054a490>
2025-12-05 20:03:03 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:03 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:03 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:03:03 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:03 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:03 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:03 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_e29098f8-163d-57fc-a241-ece190617f9c/e29098f8-163d-57fc-a241-ece190617f9c_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F280%252Fda713551a2b34e23b198e7a29daa9bdb%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:03 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/280/da713551a2b34e23b198e7a29daa9bdb/images/large/1.jpg saved to cache
2025-12-05 20:03:03 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item e29098f8-163d-57fc-a241-ece190617f9c
2025-12-05 20:03:03 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: e29098f8-163d-57fc-a241-ece190617f9c, skipping ID generation
2025-12-05 20:03:03 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e29098f8-163d-57fc-a241-ece190617f9c with scrape type 1
2025-12-05 20:03:03 [crawlers.middlewares.photo_download_middleware] INFO: Removing item e29098f8-163d-57fc-a241-ece190617f9c from tracking dictionaries
2025-12-05 20:03:03 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item e29098f8-163d-57fc-a241-ece190617f9c sending to next pipeline
2025-12-05 20:03:03 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: e29098f8-163d-57fc-a241-ece190617f9c, identifier: 1788900
2025-12-05 20:03:03 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:03 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:03 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:03 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0c74350>
2025-12-05 20:03:03 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:03 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:03 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:03:03 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:04 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:04 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:04 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_f1a13805-f134-5a1d-82ee-434a1c8ce9a0/f1a13805-f134-5a1d-82ee-434a1c8ce9a0_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F275%252Fbd636151ee6446609e38e6e3213fb46d%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:04 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/275/bd636151ee6446609e38e6e3213fb46d/images/large/1.jpg saved to cache
2025-12-05 20:03:04 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item f1a13805-f134-5a1d-82ee-434a1c8ce9a0
2025-12-05 20:03:04 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: f1a13805-f134-5a1d-82ee-434a1c8ce9a0, skipping ID generation
2025-12-05 20:03:04 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f1a13805-f134-5a1d-82ee-434a1c8ce9a0 with scrape type 1
2025-12-05 20:03:04 [crawlers.middlewares.photo_download_middleware] INFO: Removing item f1a13805-f134-5a1d-82ee-434a1c8ce9a0 from tracking dictionaries
2025-12-05 20:03:04 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item f1a13805-f134-5a1d-82ee-434a1c8ce9a0 sending to next pipeline
2025-12-05 20:03:04 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: f1a13805-f134-5a1d-82ee-434a1c8ce9a0, identifier: 1788901
2025-12-05 20:03:04 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: e29098f8-163d-57fc-a241-ece190617f9c with identifier: 1788900
2025-12-05 20:03:04 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: e29098f8-163d-57fc-a241-ece190617f9c with identifier: 1788900
2025-12-05 20:03:04 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: e29098f8-163d-57fc-a241-ece190617f9c with identifier: 1788900 to the API
2025-12-05 20:03:04 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:04 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:04 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/280/da713551a2b34e23b198e7a29daa9bdb/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 98,
 'color': 'Azul',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'e29098f8-163d-57fc-a241-ece190617f9c',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'AUDI',
 'model': 'A4',
 'odometer': 66590,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_e29098f8-163d-57fc-a241-ece190617f9c/e29098f8-163d-57fc-a241-ece190617f9c_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 376415,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/280/da713551a2b34e23b198e7a29daa9bdb/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-11-10',
 'seats_number': 5,
 'seller_name': None,
 'title': 'AUDI A4 Advanced 35 TDI 120kW (163CV) S tronic',
 'transmission': 'manual',
 'trim': 'Advanced 35 TDI 120kW (163CV) S tronic',
 'vin': None,
 'year': 2020}
2025-12-05 20:03:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788900%27 HTTP/1.1" 200 None
2025-12-05 20:03:04 [ayvens.es] INFO: Saving data for 1788900: {'created_time': 1764964984.605238, 'last_price_update_time': 1764964984.605247, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788900') HTTP/1.1" 204 0
2025-12-05 20:03:04 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: f1a13805-f134-5a1d-82ee-434a1c8ce9a0 with identifier: 1788901
2025-12-05 20:03:04 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:03:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:03:04 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: f1a13805-f134-5a1d-82ee-434a1c8ce9a0 with identifier: 1788901
2025-12-05 20:03:04 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: f1a13805-f134-5a1d-82ee-434a1c8ce9a0 with identifier: 1788901 to the API
2025-12-05 20:03:04 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:04 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:04 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/275/bd636151ee6446609e38e6e3213fb46d/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 118,
 'color': 'Platinum',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '0.9L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': 'f1a13805-f134-5a1d-82ee-434a1c8ce9a0',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'NISSAN',
 'model': 'MICRA',
 'odometer': 69671,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_f1a13805-f134-5a1d-82ee-434a1c8ce9a0/f1a13805-f134-5a1d-82ee-434a1c8ce9a0_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 318863,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/275/bd636151ee6446609e38e6e3213fb46d/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-01-28',
 'seats_number': 5,
 'seller_name': None,
 'title': 'NISSAN MICRA IG-T 66 kW (90 CV) S&S Acenta',
 'transmission': 'manual',
 'trim': 'IG-T 66 kW (90 CV) S&S Acenta',
 'vin': None,
 'year': 2019}
2025-12-05 20:03:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788901%27 HTTP/1.1" 200 None
2025-12-05 20:03:05 [ayvens.es] INFO: Saving data for 1788901: {'created_time': 1764964985.002559, 'last_price_update_time': 1764964985.00257, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788901') HTTP/1.1" 204 0
2025-12-05 20:03:05 [ayvens.es] INFO: Found listing with ID: 1788909
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788909%27 HTTP/1.1" 200 None
2025-12-05 20:03:05 [ayvens.es] INFO: Scrape type for 1788909: 1
2025-12-05 20:03:05 [ayvens.es] INFO: Extracting data for car ID: 1788909
2025-12-05 20:03:05 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: a774743d-87f6-5275-b9bd-2d128b266620 with identifier: 1788909
2025-12-05 20:03:05 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a774743d-87f6-5275-b9bd-2d128b266620 with scrape type 1
2025-12-05 20:03:05 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item a774743d-87f6-5275-b9bd-2d128b266620
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F281%252F18f9a6c427704afaa51a628afd8c4b7a%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:05 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/281/18f9a6c427704afaa51a628afd8c4b7a/images/large/1.jpg not downloaded yet
2025-12-05 20:03:05 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/352/966c19b0779e4ac682a423bca0e2d799/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:05 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:05 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:05 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:05 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b045bc90>
2025-12-05 20:03:05 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1971
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:05 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_9fac1d64-584c-5fa9-928d-807fdf792dd2/9fac1d64-584c-5fa9-928d-807fdf792dd2_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F528%252F94a119b1869340e196661ad2c7fb3e16%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:05 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/528/94a119b1869340e196661ad2c7fb3e16/images/large/1.jpg saved to cache
2025-12-05 20:03:05 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 9fac1d64-584c-5fa9-928d-807fdf792dd2
2025-12-05 20:03:05 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 9fac1d64-584c-5fa9-928d-807fdf792dd2, skipping ID generation
2025-12-05 20:03:05 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9fac1d64-584c-5fa9-928d-807fdf792dd2 with scrape type 1
2025-12-05 20:03:05 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 9fac1d64-584c-5fa9-928d-807fdf792dd2 from tracking dictionaries
2025-12-05 20:03:05 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 9fac1d64-584c-5fa9-928d-807fdf792dd2 sending to next pipeline
2025-12-05 20:03:05 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 9fac1d64-584c-5fa9-928d-807fdf792dd2, identifier: 1788902
2025-12-05 20:03:05 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/672/414c295b73d842b995ad8d13d5cb7c35/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:05 [ayvens.es] INFO: Found listing with ID: 1788910
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788910%27 HTTP/1.1" 200 None
2025-12-05 20:03:05 [ayvens.es] INFO: Scrape type for 1788910: 1
2025-12-05 20:03:05 [ayvens.es] INFO: Extracting data for car ID: 1788910
2025-12-05 20:03:05 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 649ebc19-9906-5e07-b1d9-b445a72ea78b with identifier: 1788910
2025-12-05 20:03:05 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 649ebc19-9906-5e07-b1d9-b445a72ea78b with scrape type 1
2025-12-05 20:03:05 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 649ebc19-9906-5e07-b1d9-b445a72ea78b
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F319%252F4f730cd504ac43fdbea18b0c93babd4b%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:05 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/319/4f730cd504ac43fdbea18b0c93babd4b/images/large/1.jpg not downloaded yet
2025-12-05 20:03:05 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 9fac1d64-584c-5fa9-928d-807fdf792dd2 with identifier: 1788902
2025-12-05 20:03:05 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:03:05 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 9fac1d64-584c-5fa9-928d-807fdf792dd2 with identifier: 1788902
2025-12-05 20:03:05 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: 153884 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    153884
2025-12-05 20:03:05 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE_FOR_COUNTRY failed for item: 9fac1d64-584c-5fa9-928d-807fdf792dd2 spider: ayvens.es
2025-12-05 20:03:05 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 9fac1d64-584c-5fa9-928d-807fdf792dd2 due to rule: high_mileage_for_country.json, error: 153884 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    153884
{'auction_title': '',
 'c_o_2_emission_value': 123,
 'color': 'Bronce',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '9fac1d64-584c-5fa9-928d-807fdf792dd2',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'NISSAN',
 'model': 'JUKE',
 'odometer': 153884,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_9fac1d64-584c-5fa9-928d-807fdf792dd2/9fac1d64-584c-5fa9-928d-807fdf792dd2_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 345011,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/528/94a119b1869340e196661ad2c7fb3e16/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2018-11-20',
 'seats_number': 5,
 'seller_name': None,
 'title': 'NISSAN JUKE dCi E6C 81 kW (110 CV) 6M/T N-CONNECTA',
 'transmission': 'manual',
 'trim': 'dCi E6C 81 kW (110 CV) 6M/T N-CONNECTA',
 'vin': None,
 'year': 2018}
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788902%27 HTTP/1.1" 200 None
2025-12-05 20:03:05 [ayvens.es] INFO: Saving data for 1788902: {'created_time': 1764964985.825543, 'last_price_update_time': 1764964985.825556, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788902') HTTP/1.1" 204 0
2025-12-05 20:03:05 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/725/c1c0d3db26eb4bcb8341e192fbcc1ebe/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:05 [ayvens.es] INFO: Found listing with ID: 1789055
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789055%27 HTTP/1.1" 200 None
2025-12-05 20:03:05 [ayvens.es] INFO: Scrape type for 1789055: 1
2025-12-05 20:03:05 [ayvens.es] INFO: Extracting data for car ID: 1789055
2025-12-05 20:03:05 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: e9e09747-ab4f-54ee-b7ad-8e831ab5eac5 with identifier: 1789055
2025-12-05 20:03:05 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e9e09747-ab4f-54ee-b7ad-8e831ab5eac5 with scrape type 1
2025-12-05 20:03:05 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item e9e09747-ab4f-54ee-b7ad-8e831ab5eac5
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F532%252F538006286a214a23b3bdb57e631916c7%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:05 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/532/538006286a214a23b3bdb57e631916c7/images/large/1.jpg not downloaded yet
2025-12-05 20:03:05 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/806/6f8afbbf23554fa385b64cf5e11c2dca/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:05 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:05 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:05 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:05 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0524450>
2025-12-05 20:03:05 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:05 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:06 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:03:06 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:06 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:06 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:06 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_e9f87121-c76a-5536-bd4a-4072ebb11155/e9f87121-c76a-5536-bd4a-4072ebb11155_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F352%252F966c19b0779e4ac682a423bca0e2d799%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:06 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/352/966c19b0779e4ac682a423bca0e2d799/images/large/1.jpg saved to cache
2025-12-05 20:03:06 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item e9f87121-c76a-5536-bd4a-4072ebb11155
2025-12-05 20:03:06 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: e9f87121-c76a-5536-bd4a-4072ebb11155, skipping ID generation
2025-12-05 20:03:06 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e9f87121-c76a-5536-bd4a-4072ebb11155 with scrape type 1
2025-12-05 20:03:06 [crawlers.middlewares.photo_download_middleware] INFO: Removing item e9f87121-c76a-5536-bd4a-4072ebb11155 from tracking dictionaries
2025-12-05 20:03:06 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item e9f87121-c76a-5536-bd4a-4072ebb11155 sending to next pipeline
2025-12-05 20:03:06 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: e9f87121-c76a-5536-bd4a-4072ebb11155, identifier: 1788903
2025-12-05 20:03:06 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:06 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:06 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:06 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0403690>
2025-12-05 20:03:06 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:06 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:06 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:03:06 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:06 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:06 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:06 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_9eec7ff8-1586-5785-ac80-1ec043fc4596/9eec7ff8-1586-5785-ac80-1ec043fc4596_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F672%252F414c295b73d842b995ad8d13d5cb7c35%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:06 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/672/414c295b73d842b995ad8d13d5cb7c35/images/large/1.jpg saved to cache
2025-12-05 20:03:06 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 9eec7ff8-1586-5785-ac80-1ec043fc4596
2025-12-05 20:03:06 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 9eec7ff8-1586-5785-ac80-1ec043fc4596, skipping ID generation
2025-12-05 20:03:06 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9eec7ff8-1586-5785-ac80-1ec043fc4596 with scrape type 1
2025-12-05 20:03:06 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 9eec7ff8-1586-5785-ac80-1ec043fc4596 from tracking dictionaries
2025-12-05 20:03:06 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 9eec7ff8-1586-5785-ac80-1ec043fc4596 sending to next pipeline
2025-12-05 20:03:06 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 9eec7ff8-1586-5785-ac80-1ec043fc4596, identifier: 1788904
2025-12-05 20:03:06 [ayvens.es] INFO: Found listing with ID: 1789056
2025-12-05 20:03:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789056%27 HTTP/1.1" 200 None
2025-12-05 20:03:06 [ayvens.es] INFO: Scrape type for 1789056: 1
2025-12-05 20:03:06 [ayvens.es] INFO: Extracting data for car ID: 1789056
2025-12-05 20:03:06 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 357af4b6-71c9-522f-aca6-a0068a202fb1 with identifier: 1789056
2025-12-05 20:03:06 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 357af4b6-71c9-522f-aca6-a0068a202fb1 with scrape type 1
2025-12-05 20:03:06 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 357af4b6-71c9-522f-aca6-a0068a202fb1
2025-12-05 20:03:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F810%252F6fa512605b1543f88cd6ef5dc2949f83%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:06 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/810/6fa512605b1543f88cd6ef5dc2949f83/images/large/1.jpg not downloaded yet
2025-12-05 20:03:06 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: e9f87121-c76a-5536-bd4a-4072ebb11155 with identifier: 1788903
2025-12-05 20:03:06 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: e9f87121-c76a-5536-bd4a-4072ebb11155 with identifier: 1788903
2025-12-05 20:03:07 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: e9f87121-c76a-5536-bd4a-4072ebb11155 with identifier: 1788903 to the API
2025-12-05 20:03:07 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:07 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:07 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/352/966c19b0779e4ac682a423bca0e2d799/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 144,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': 'e9f87121-c76a-5536-bd4a-4072ebb11155',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SKODA',
 'model': 'KAROQ',
 'odometer': 108158,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_e9f87121-c76a-5536-bd4a-4072ebb11155/e9f87121-c76a-5536-bd4a-4072ebb11155_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 505600,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/352/966c19b0779e4ac682a423bca0e2d799/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-06-17',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SKODA KAROQ 1.5 TSI 110kW (150CV) DSG ACT Ambition',
 'transmission': 'manual',
 'trim': '1.5 TSI 110kW (150CV) DSG ACT Ambition',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788903%27 HTTP/1.1" 200 None
2025-12-05 20:03:07 [ayvens.es] INFO: Saving data for 1788903: {'created_time': 1764964987.305211, 'last_price_update_time': 1764964987.305222, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788903') HTTP/1.1" 204 0
2025-12-05 20:03:07 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 9eec7ff8-1586-5785-ac80-1ec043fc4596 with identifier: 1788904
2025-12-05 20:03:07 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 9eec7ff8-1586-5785-ac80-1ec043fc4596 with identifier: 1788904
2025-12-05 20:03:07 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 9eec7ff8-1586-5785-ac80-1ec043fc4596 with identifier: 1788904 to the API
2025-12-05 20:03:07 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:07 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:07 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/672/414c295b73d842b995ad8d13d5cb7c35/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 112,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '9eec7ff8-1586-5785-ac80-1ec043fc4596',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'NISSAN',
 'model': 'JUKE',
 'odometer': 86179,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_9eec7ff8-1586-5785-ac80-1ec043fc4596/9eec7ff8-1586-5785-ac80-1ec043fc4596_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 344773,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/672/414c295b73d842b995ad8d13d5cb7c35/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-08-06',
 'seats_number': 5,
 'seller_name': None,
 'title': 'NISSAN JUKE DIG-T 86 kW (117 CV) 6 M/T ACENTA',
 'transmission': 'manual',
 'trim': 'DIG-T 86 kW (117 CV) 6 M/T ACENTA',
 'vin': None,
 'year': 2020}
2025-12-05 20:03:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788904%27 HTTP/1.1" 200 None
2025-12-05 20:03:07 [ayvens.es] INFO: Saving data for 1788904: {'created_time': 1764964987.748793, 'last_price_update_time': 1764964987.748806, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788904') HTTP/1.1" 204 0
2025-12-05 20:03:07 [scrapy.extensions.logstats] INFO: Crawled 5 pages (at 1 pages/min), scraped 105 items (at 47 items/min)
2025-12-05 20:03:07 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/594/2ea163777fca41ccba428d8a468bb363/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:07 [ayvens.es] INFO: Found listing with ID: 1789057
2025-12-05 20:03:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789057%27 HTTP/1.1" 200 None
2025-12-05 20:03:07 [ayvens.es] INFO: Scrape type for 1789057: 1
2025-12-05 20:03:07 [ayvens.es] INFO: Extracting data for car ID: 1789057
2025-12-05 20:03:07 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 52758577-3d85-5b21-b21e-31b549b36868 with identifier: 1789057
2025-12-05 20:03:07 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 52758577-3d85-5b21-b21e-31b549b36868 with scrape type 1
2025-12-05 20:03:07 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 52758577-3d85-5b21-b21e-31b549b36868
2025-12-05 20:03:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F647%252F53486f6a82214db4a72d15579ea5167a%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:07 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/647/53486f6a82214db4a72d15579ea5167a/images/large/1.jpg not downloaded yet
2025-12-05 20:03:07 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/534/fd7541a47881435b8cf3f6ece86e0247/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:07 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:07 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:07 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:07 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0cf9390>
2025-12-05 20:03:07 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:07 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:08 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:03:08 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:08 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:08 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:08 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_411b2b41-33e5-5062-863e-a9eabde3e8e9/411b2b41-33e5-5062-863e-a9eabde3e8e9_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:08 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:08 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F725%252Fc1c0d3db26eb4bcb8341e192fbcc1ebe%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:08 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/725/c1c0d3db26eb4bcb8341e192fbcc1ebe/images/large/1.jpg saved to cache
2025-12-05 20:03:08 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 411b2b41-33e5-5062-863e-a9eabde3e8e9
2025-12-05 20:03:08 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 411b2b41-33e5-5062-863e-a9eabde3e8e9, skipping ID generation
2025-12-05 20:03:08 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 411b2b41-33e5-5062-863e-a9eabde3e8e9 with scrape type 1
2025-12-05 20:03:08 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 411b2b41-33e5-5062-863e-a9eabde3e8e9 from tracking dictionaries
2025-12-05 20:03:08 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 411b2b41-33e5-5062-863e-a9eabde3e8e9 sending to next pipeline
2025-12-05 20:03:08 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 411b2b41-33e5-5062-863e-a9eabde3e8e9, identifier: 1788905
2025-12-05 20:03:08 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:08 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:08 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:08 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0dc32d0>
2025-12-05 20:03:08 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:08 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:08 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:03:08 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:08 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:08 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:08 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_f32ffefe-4ebb-58e3-942b-e5a2681ac5e3/f32ffefe-4ebb-58e3-942b-e5a2681ac5e3_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:08 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:08 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F806%252F6f8afbbf23554fa385b64cf5e11c2dca%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:08 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/806/6f8afbbf23554fa385b64cf5e11c2dca/images/large/1.jpg saved to cache
2025-12-05 20:03:08 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item f32ffefe-4ebb-58e3-942b-e5a2681ac5e3
2025-12-05 20:03:08 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: f32ffefe-4ebb-58e3-942b-e5a2681ac5e3, skipping ID generation
2025-12-05 20:03:08 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f32ffefe-4ebb-58e3-942b-e5a2681ac5e3 with scrape type 1
2025-12-05 20:03:08 [crawlers.middlewares.photo_download_middleware] INFO: Removing item f32ffefe-4ebb-58e3-942b-e5a2681ac5e3 from tracking dictionaries
2025-12-05 20:03:08 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item f32ffefe-4ebb-58e3-942b-e5a2681ac5e3 sending to next pipeline
2025-12-05 20:03:08 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: f32ffefe-4ebb-58e3-942b-e5a2681ac5e3, identifier: 1788906
2025-12-05 20:03:08 [ayvens.es] INFO: Found listing with ID: 1789058
2025-12-05 20:03:08 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:08 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789058%27 HTTP/1.1" 200 None
2025-12-05 20:03:08 [ayvens.es] INFO: Scrape type for 1789058: 1
2025-12-05 20:03:08 [ayvens.es] INFO: Extracting data for car ID: 1789058
2025-12-05 20:03:08 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 8a040621-320e-5d5e-9e85-3d98feb9cdee with identifier: 1789058
2025-12-05 20:03:08 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 8a040621-320e-5d5e-9e85-3d98feb9cdee with scrape type 1
2025-12-05 20:03:08 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 8a040621-320e-5d5e-9e85-3d98feb9cdee
2025-12-05 20:03:08 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:08 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F647%252F5aa5e9c6b96248aca99f50ee960c46df%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:08 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/647/5aa5e9c6b96248aca99f50ee960c46df/images/large/1.jpg not downloaded yet
2025-12-05 20:03:08 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 411b2b41-33e5-5062-863e-a9eabde3e8e9 with identifier: 1788905
2025-12-05 20:03:08 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 411b2b41-33e5-5062-863e-a9eabde3e8e9 with identifier: 1788905
2025-12-05 20:03:09 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 411b2b41-33e5-5062-863e-a9eabde3e8e9 with identifier: 1788905 to the API
2025-12-05 20:03:09 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:09 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:09 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/725/c1c0d3db26eb4bcb8341e192fbcc1ebe/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 108,
 'color': 'Mineralgrau',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '411b2b41-33e5-5062-863e-a9eabde3e8e9',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'BMW',
 'model': 'SERIES 2 GRAN TOURER',
 'odometer': 85257,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_411b2b41-33e5-5062-863e-a9eabde3e8e9/411b2b41-33e5-5062-863e-a9eabde3e8e9_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 380107,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/725/c1c0d3db26eb4bcb8341e192fbcc1ebe/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-11-06',
 'seats_number': 5,
 'seller_name': None,
 'title': 'BMW SERIES 2 GRAN TOURER 216d',
 'transmission': 'manual',
 'trim': '216d',
 'vin': None,
 'year': 2020}
2025-12-05 20:03:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788905%27 HTTP/1.1" 200 None
2025-12-05 20:03:09 [ayvens.es] INFO: Saving data for 1788905: {'created_time': 1764964989.291022, 'last_price_update_time': 1764964989.291033, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788905') HTTP/1.1" 204 0
2025-12-05 20:03:09 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: f32ffefe-4ebb-58e3-942b-e5a2681ac5e3 with identifier: 1788906
2025-12-05 20:03:09 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: f32ffefe-4ebb-58e3-942b-e5a2681ac5e3 with identifier: 1788906
2025-12-05 20:03:09 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: f32ffefe-4ebb-58e3-942b-e5a2681ac5e3 with identifier: 1788906 to the API
2025-12-05 20:03:09 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:09 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:09 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/806/6f8afbbf23554fa385b64cf5e11c2dca/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 129,
 'color': 'Gris',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': 'f32ffefe-4ebb-58e3-942b-e5a2681ac5e3',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'TIGUAN',
 'odometer': 148457,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_f32ffefe-4ebb-58e3-942b-e5a2681ac5e3/f32ffefe-4ebb-58e3-942b-e5a2681ac5e3_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 330214,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/806/6f8afbbf23554fa385b64cf5e11c2dca/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-08-06',
 'seats_number': 5,
 'seller_name': None,
 'title': 'VOLKSWAGEN TIGUAN Sport 1.5 TSI 110kW (150CV) DSG',
 'transmission': 'manual',
 'trim': 'Sport 1.5 TSI 110kW (150CV) DSG',
 'vin': None,
 'year': 2020}
2025-12-05 20:03:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788906%27 HTTP/1.1" 200 None
2025-12-05 20:03:09 [ayvens.es] INFO: Saving data for 1788906: {'created_time': 1764964989.661442, 'last_price_update_time': 1764964989.661453, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788906') HTTP/1.1" 204 0
2025-12-05 20:03:09 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/319/4f730cd504ac43fdbea18b0c93babd4b/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:09 [ayvens.es] INFO: Found listing with ID: 1789059
2025-12-05 20:03:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789059%27 HTTP/1.1" 200 None
2025-12-05 20:03:09 [ayvens.es] INFO: Scrape type for 1789059: 1
2025-12-05 20:03:09 [ayvens.es] INFO: Extracting data for car ID: 1789059
2025-12-05 20:03:09 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 3cf4cdd2-94e2-5487-9768-c4b9b4fa5408 with identifier: 1789059
2025-12-05 20:03:09 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 3cf4cdd2-94e2-5487-9768-c4b9b4fa5408 with scrape type 1
2025-12-05 20:03:09 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 3cf4cdd2-94e2-5487-9768-c4b9b4fa5408
2025-12-05 20:03:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:09 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F104%252F1275ac45aac54053bcfea51f6dacd02e%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:09 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/104/1275ac45aac54053bcfea51f6dacd02e/images/large/1.jpg not downloaded yet
2025-12-05 20:03:09 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/281/18f9a6c427704afaa51a628afd8c4b7a/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:09 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:09 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:09 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:09 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b473d0>
2025-12-05 20:03:09 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:09 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:09 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:03:09 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:10 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:10 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:10 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_63002157-1c41-58c1-9117-b75b5574d5be/63002157-1c41-58c1-9117-b75b5574d5be_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F594%252F2ea163777fca41ccba428d8a468bb363%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:10 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/594/2ea163777fca41ccba428d8a468bb363/images/large/1.jpg saved to cache
2025-12-05 20:03:10 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 63002157-1c41-58c1-9117-b75b5574d5be
2025-12-05 20:03:10 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 63002157-1c41-58c1-9117-b75b5574d5be, skipping ID generation
2025-12-05 20:03:10 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 63002157-1c41-58c1-9117-b75b5574d5be with scrape type 1
2025-12-05 20:03:10 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 63002157-1c41-58c1-9117-b75b5574d5be from tracking dictionaries
2025-12-05 20:03:10 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 63002157-1c41-58c1-9117-b75b5574d5be sending to next pipeline
2025-12-05 20:03:10 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 63002157-1c41-58c1-9117-b75b5574d5be, identifier: 1788907
2025-12-05 20:03:10 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:10 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:10 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:10 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a82350>
2025-12-05 20:03:10 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:10 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:10 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:03:10 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:10 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:10 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:10 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_15326f39-ea91-5b1c-8f72-8155a26e072f/15326f39-ea91-5b1c-8f72-8155a26e072f_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F534%252Ffd7541a47881435b8cf3f6ece86e0247%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:10 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/534/fd7541a47881435b8cf3f6ece86e0247/images/large/1.jpg saved to cache
2025-12-05 20:03:10 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 15326f39-ea91-5b1c-8f72-8155a26e072f
2025-12-05 20:03:10 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 15326f39-ea91-5b1c-8f72-8155a26e072f, skipping ID generation
2025-12-05 20:03:10 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 15326f39-ea91-5b1c-8f72-8155a26e072f with scrape type 1
2025-12-05 20:03:10 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 15326f39-ea91-5b1c-8f72-8155a26e072f from tracking dictionaries
2025-12-05 20:03:10 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 15326f39-ea91-5b1c-8f72-8155a26e072f sending to next pipeline
2025-12-05 20:03:10 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 15326f39-ea91-5b1c-8f72-8155a26e072f, identifier: 1788908
2025-12-05 20:03:10 [ayvens.es] INFO: Found listing with ID: 1789060
2025-12-05 20:03:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789060%27 HTTP/1.1" 200 None
2025-12-05 20:03:10 [ayvens.es] INFO: Scrape type for 1789060: 1
2025-12-05 20:03:10 [ayvens.es] INFO: Extracting data for car ID: 1789060
2025-12-05 20:03:10 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 485aee94-af0d-5e1c-886e-304a791500a4 with identifier: 1789060
2025-12-05 20:03:10 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 485aee94-af0d-5e1c-886e-304a791500a4 with scrape type 1
2025-12-05 20:03:10 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 485aee94-af0d-5e1c-886e-304a791500a4
2025-12-05 20:03:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:10 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F677%252F1e76ebcdd95545e1a9eddbbc61b97138%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:10 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/677/1e76ebcdd95545e1a9eddbbc61b97138/images/large/1.jpg not downloaded yet
2025-12-05 20:03:10 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 63002157-1c41-58c1-9117-b75b5574d5be with identifier: 1788907
2025-12-05 20:03:10 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 63002157-1c41-58c1-9117-b75b5574d5be with identifier: 1788907
2025-12-05 20:03:10 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 63002157-1c41-58c1-9117-b75b5574d5be with identifier: 1788907 to the API
2025-12-05 20:03:10 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:11 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:11 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/594/2ea163777fca41ccba428d8a468bb363/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 120,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '63002157-1c41-58c1-9117-b75b5574d5be',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'SEAT',
 'model': 'ARONA',
 'odometer': 16992,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_63002157-1c41-58c1-9117-b75b5574d5be/63002157-1c41-58c1-9117-b75b5574d5be_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 346839,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/594/2ea163777fca41ccba428d8a468bb363/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-11-26',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SEAT ARONA 1.0 TSI 81kW (110CV) Style',
 'transmission': 'manual',
 'trim': '1.0 TSI 81kW (110CV) Style',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788907%27 HTTP/1.1" 200 None
2025-12-05 20:03:11 [ayvens.es] INFO: Saving data for 1788907: {'created_time': 1764964991.197996, 'last_price_update_time': 1764964991.19801, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788907') HTTP/1.1" 204 0
2025-12-05 20:03:11 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 15326f39-ea91-5b1c-8f72-8155a26e072f with identifier: 1788908
2025-12-05 20:03:11 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 15326f39-ea91-5b1c-8f72-8155a26e072f with identifier: 1788908
2025-12-05 20:03:11 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 15326f39-ea91-5b1c-8f72-8155a26e072f with identifier: 1788908 to the API
2025-12-05 20:03:11 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:11 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:11 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/534/fd7541a47881435b8cf3f6ece86e0247/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 121,
 'color': 'Azul',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '15326f39-ea91-5b1c-8f72-8155a26e072f',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'FORD',
 'model': 'PUMA',
 'odometer': 91534,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_15326f39-ea91-5b1c-8f72-8155a26e072f/15326f39-ea91-5b1c-8f72-8155a26e072f_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 386925,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/534/fd7541a47881435b8cf3f6ece86e0247/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-09-30',
 'seats_number': 5,
 'seller_name': None,
 'title': 'FORD PUMA 1.0 EcoBoost 125cv ST-Line X MHEV',
 'transmission': 'manual',
 'trim': '1.0 EcoBoost 125cv ST-Line X MHEV',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788908%27 HTTP/1.1" 200 None
2025-12-05 20:03:11 [ayvens.es] INFO: Saving data for 1788908: {'created_time': 1764964991.659711, 'last_price_update_time': 1764964991.65972, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788908') HTTP/1.1" 204 0
2025-12-05 20:03:11 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/532/538006286a214a23b3bdb57e631916c7/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:11 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/810/6fa512605b1543f88cd6ef5dc2949f83/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:11 [ayvens.es] INFO: Found listing with ID: 1789061
2025-12-05 20:03:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789061%27 HTTP/1.1" 200 None
2025-12-05 20:03:11 [ayvens.es] INFO: Scrape type for 1789061: 1
2025-12-05 20:03:11 [ayvens.es] INFO: Extracting data for car ID: 1789061
2025-12-05 20:03:11 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: bfd49ef3-42d9-51aa-9a0b-c46efa552816 with identifier: 1789061
2025-12-05 20:03:11 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item bfd49ef3-42d9-51aa-9a0b-c46efa552816 with scrape type 1
2025-12-05 20:03:11 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item bfd49ef3-42d9-51aa-9a0b-c46efa552816
2025-12-05 20:03:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F294%252Fc4127973473c4b708d117cb6e72318c4%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:11 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/294/c4127973473c4b708d117cb6e72318c4/images/large/1.jpg not downloaded yet
2025-12-05 20:03:11 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:11 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:11 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:11 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0aa71d0>
2025-12-05 20:03:11 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:11 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:11 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:03:11 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:12 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:12 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:12 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_649ebc19-9906-5e07-b1d9-b445a72ea78b/649ebc19-9906-5e07-b1d9-b445a72ea78b_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F319%252F4f730cd504ac43fdbea18b0c93babd4b%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:12 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/319/4f730cd504ac43fdbea18b0c93babd4b/images/large/1.jpg saved to cache
2025-12-05 20:03:12 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 649ebc19-9906-5e07-b1d9-b445a72ea78b
2025-12-05 20:03:12 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 649ebc19-9906-5e07-b1d9-b445a72ea78b, skipping ID generation
2025-12-05 20:03:12 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 649ebc19-9906-5e07-b1d9-b445a72ea78b with scrape type 1
2025-12-05 20:03:12 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 649ebc19-9906-5e07-b1d9-b445a72ea78b from tracking dictionaries
2025-12-05 20:03:12 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 649ebc19-9906-5e07-b1d9-b445a72ea78b sending to next pipeline
2025-12-05 20:03:12 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 649ebc19-9906-5e07-b1d9-b445a72ea78b, identifier: 1788910
2025-12-05 20:03:12 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:12 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:12 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:12 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b04cd250>
2025-12-05 20:03:12 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:12 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:12 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:03:12 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:12 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:12 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:12 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_a774743d-87f6-5275-b9bd-2d128b266620/a774743d-87f6-5275-b9bd-2d128b266620_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F281%252F18f9a6c427704afaa51a628afd8c4b7a%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:12 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/281/18f9a6c427704afaa51a628afd8c4b7a/images/large/1.jpg saved to cache
2025-12-05 20:03:12 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item a774743d-87f6-5275-b9bd-2d128b266620
2025-12-05 20:03:12 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: a774743d-87f6-5275-b9bd-2d128b266620, skipping ID generation
2025-12-05 20:03:12 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a774743d-87f6-5275-b9bd-2d128b266620 with scrape type 1
2025-12-05 20:03:12 [crawlers.middlewares.photo_download_middleware] INFO: Removing item a774743d-87f6-5275-b9bd-2d128b266620 from tracking dictionaries
2025-12-05 20:03:12 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item a774743d-87f6-5275-b9bd-2d128b266620 sending to next pipeline
2025-12-05 20:03:12 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: a774743d-87f6-5275-b9bd-2d128b266620, identifier: 1788909
2025-12-05 20:03:12 [ayvens.es] INFO: Found listing with ID: 1789062
2025-12-05 20:03:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789062%27 HTTP/1.1" 200 None
2025-12-05 20:03:12 [ayvens.es] INFO: Scrape type for 1789062: 1
2025-12-05 20:03:12 [ayvens.es] INFO: Extracting data for car ID: 1789062
2025-12-05 20:03:12 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 10e434db-19c6-5f46-b867-84c7c701fa24 with identifier: 1789062
2025-12-05 20:03:12 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 10e434db-19c6-5f46-b867-84c7c701fa24 with scrape type 1
2025-12-05 20:03:12 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 10e434db-19c6-5f46-b867-84c7c701fa24
2025-12-05 20:03:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F770%252F4dc77b39377e4b21a9104bb7d9d93472%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:12 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/770/4dc77b39377e4b21a9104bb7d9d93472/images/large/1.jpg not downloaded yet
2025-12-05 20:03:12 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 649ebc19-9906-5e07-b1d9-b445a72ea78b with identifier: 1788910
2025-12-05 20:03:12 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 649ebc19-9906-5e07-b1d9-b445a72ea78b with identifier: 1788910
2025-12-05 20:03:12 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 649ebc19-9906-5e07-b1d9-b445a72ea78b with identifier: 1788910 to the API
2025-12-05 20:03:12 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:13 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:13 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/319/4f730cd504ac43fdbea18b0c93babd4b/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 114,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '649ebc19-9906-5e07-b1d9-b445a72ea78b',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'GOLF',
 'odometer': 103437,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_649ebc19-9906-5e07-b1d9-b445a72ea78b/649ebc19-9906-5e07-b1d9-b445a72ea78b_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 316276,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/319/4f730cd504ac43fdbea18b0c93babd4b/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-07-14',
 'seats_number': 5,
 'seller_name': None,
 'title': 'VOLKSWAGEN GOLF Advance 2.0 TDI 110kW (150CV)',
 'transmission': 'manual',
 'trim': 'Advance 2.0 TDI 110kW (150CV)',
 'vin': None,
 'year': 2020}
2025-12-05 20:03:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788910%27 HTTP/1.1" 200 None
2025-12-05 20:03:13 [ayvens.es] INFO: Saving data for 1788910: {'created_time': 1764964993.198551, 'last_price_update_time': 1764964993.198563, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788910') HTTP/1.1" 204 0
2025-12-05 20:03:13 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: a774743d-87f6-5275-b9bd-2d128b266620 with identifier: 1788909
2025-12-05 20:03:13 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: a774743d-87f6-5275-b9bd-2d128b266620 with identifier: 1788909
2025-12-05 20:03:13 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: a774743d-87f6-5275-b9bd-2d128b266620 with identifier: 1788909 to the API
2025-12-05 20:03:13 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:13 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:13 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/281/18f9a6c427704afaa51a628afd8c4b7a/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 106,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'a774743d-87f6-5275-b9bd-2d128b266620',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37763',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'CLIO',
 'odometer': 72417,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_a774743d-87f6-5275-b9bd-2d128b266620/a774743d-87f6-5275-b9bd-2d128b266620_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 417891,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/281/18f9a6c427704afaa51a628afd8c4b7a/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-05-13',
 'seats_number': 5,
 'seller_name': None,
 'title': 'RENAULT CLIO Business dCi 66kW (90CV) -18',
 'transmission': 'manual',
 'trim': 'Business dCi 66kW (90CV) -18',
 'vin': None,
 'year': 2019}
2025-12-05 20:03:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271788909%27 HTTP/1.1" 200 None
2025-12-05 20:03:13 [ayvens.es] INFO: Saving data for 1788909: {'created_time': 1764964993.626483, 'last_price_update_time': 1764964993.626492, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1788909') HTTP/1.1" 204 0
2025-12-05 20:03:13 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/647/53486f6a82214db4a72d15579ea5167a/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:13 [ayvens.es] INFO: Found listing with ID: 1789063
2025-12-05 20:03:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789063%27 HTTP/1.1" 200 None
2025-12-05 20:03:13 [ayvens.es] INFO: Scrape type for 1789063: 1
2025-12-05 20:03:13 [ayvens.es] INFO: Extracting data for car ID: 1789063
2025-12-05 20:03:13 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: caeb8e5a-7ad9-5b8b-ae76-687a5404796b with identifier: 1789063
2025-12-05 20:03:13 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item caeb8e5a-7ad9-5b8b-ae76-687a5404796b with scrape type 1
2025-12-05 20:03:13 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item caeb8e5a-7ad9-5b8b-ae76-687a5404796b
2025-12-05 20:03:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F407%252F7b3bff0f746d41d48db9cafa32ac96cd%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:13 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/407/7b3bff0f746d41d48db9cafa32ac96cd/images/large/1.jpg not downloaded yet
2025-12-05 20:03:13 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/647/5aa5e9c6b96248aca99f50ee960c46df/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:13 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:13 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:13 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:13 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b4ce50>
2025-12-05 20:03:13 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:13 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:13 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1969
2025-12-05 20:03:13 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:13 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:13 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:13 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_e9e09747-ab4f-54ee-b7ad-8e831ab5eac5/e9e09747-ab4f-54ee-b7ad-8e831ab5eac5_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F532%252F538006286a214a23b3bdb57e631916c7%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:14 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/532/538006286a214a23b3bdb57e631916c7/images/large/1.jpg saved to cache
2025-12-05 20:03:14 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item e9e09747-ab4f-54ee-b7ad-8e831ab5eac5
2025-12-05 20:03:14 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: e9e09747-ab4f-54ee-b7ad-8e831ab5eac5, skipping ID generation
2025-12-05 20:03:14 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e9e09747-ab4f-54ee-b7ad-8e831ab5eac5 with scrape type 1
2025-12-05 20:03:14 [crawlers.middlewares.photo_download_middleware] INFO: Removing item e9e09747-ab4f-54ee-b7ad-8e831ab5eac5 from tracking dictionaries
2025-12-05 20:03:14 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item e9e09747-ab4f-54ee-b7ad-8e831ab5eac5 sending to next pipeline
2025-12-05 20:03:14 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: e9e09747-ab4f-54ee-b7ad-8e831ab5eac5, identifier: 1789055
2025-12-05 20:03:14 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:14 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:14 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:14 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0dc02d0>
2025-12-05 20:03:14 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:14 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:14 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1969
2025-12-05 20:03:14 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:14 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:14 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:14 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_357af4b6-71c9-522f-aca6-a0068a202fb1/357af4b6-71c9-522f-aca6-a0068a202fb1_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F810%252F6fa512605b1543f88cd6ef5dc2949f83%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:14 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/810/6fa512605b1543f88cd6ef5dc2949f83/images/large/1.jpg saved to cache
2025-12-05 20:03:14 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 357af4b6-71c9-522f-aca6-a0068a202fb1
2025-12-05 20:03:14 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 357af4b6-71c9-522f-aca6-a0068a202fb1, skipping ID generation
2025-12-05 20:03:14 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 357af4b6-71c9-522f-aca6-a0068a202fb1 with scrape type 1
2025-12-05 20:03:14 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 357af4b6-71c9-522f-aca6-a0068a202fb1 from tracking dictionaries
2025-12-05 20:03:14 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 357af4b6-71c9-522f-aca6-a0068a202fb1 sending to next pipeline
2025-12-05 20:03:14 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 357af4b6-71c9-522f-aca6-a0068a202fb1, identifier: 1789056
2025-12-05 20:03:14 [ayvens.es] INFO: Found listing with ID: 1789064
2025-12-05 20:03:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789064%27 HTTP/1.1" 200 None
2025-12-05 20:03:14 [ayvens.es] INFO: Scrape type for 1789064: 1
2025-12-05 20:03:14 [ayvens.es] INFO: Extracting data for car ID: 1789064
2025-12-05 20:03:14 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 8a0c3831-cf50-5b2c-b2fe-6af14adb4f07 with identifier: 1789064
2025-12-05 20:03:14 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 8a0c3831-cf50-5b2c-b2fe-6af14adb4f07 with scrape type 1
2025-12-05 20:03:14 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 8a0c3831-cf50-5b2c-b2fe-6af14adb4f07
2025-12-05 20:03:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F190%252Fac8f475ad9994eee80c955daa92bbaae%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:14 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/190/ac8f475ad9994eee80c955daa92bbaae/images/large/1.jpg not downloaded yet
2025-12-05 20:03:14 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: e9e09747-ab4f-54ee-b7ad-8e831ab5eac5 with identifier: 1789055
2025-12-05 20:03:14 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: e9e09747-ab4f-54ee-b7ad-8e831ab5eac5 with identifier: 1789055
2025-12-05 20:03:14 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: e9e09747-ab4f-54ee-b7ad-8e831ab5eac5 with identifier: 1789055 to the API
2025-12-05 20:03:14 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:14 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:14 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/532/538006286a214a23b3bdb57e631916c7/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 156,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.9L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'e9e09747-ab4f-54ee-b7ad-8e831ab5eac5',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'MERCEDES-BENZ',
 'model': 'GLC-CLASS',
 'odometer': 148951,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_e9e09747-ab4f-54ee-b7ad-8e831ab5eac5/e9e09747-ab4f-54ee-b7ad-8e831ab5eac5_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 340551,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/532/538006286a214a23b3bdb57e631916c7/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-10-04',
 'seats_number': 5,
 'seller_name': None,
 'title': 'MERCEDES-BENZ GLC-CLASS GLC 220 d 4MATIC',
 'transmission': 'manual',
 'trim': 'GLC 220 d 4MATIC',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789055%27 HTTP/1.1" 200 None
2025-12-05 20:03:15 [ayvens.es] INFO: Saving data for 1789055: {'created_time': 1764964995.038589, 'last_price_update_time': 1764964995.038603, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789055') HTTP/1.1" 204 0
2025-12-05 20:03:15 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 357af4b6-71c9-522f-aca6-a0068a202fb1 with identifier: 1789056
2025-12-05 20:03:15 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 357af4b6-71c9-522f-aca6-a0068a202fb1 with identifier: 1789056
2025-12-05 20:03:15 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 357af4b6-71c9-522f-aca6-a0068a202fb1 with identifier: 1789056 to the API
2025-12-05 20:03:15 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:15 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:15 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/810/6fa512605b1543f88cd6ef5dc2949f83/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 117,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '357af4b6-71c9-522f-aca6-a0068a202fb1',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'CITROËN',
 'model': 'C3',
 'odometer': 144673,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_357af4b6-71c9-522f-aca6-a0068a202fb1/357af4b6-71c9-522f-aca6-a0068a202fb1_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 320094,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/810/6fa512605b1543f88cd6ef5dc2949f83/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-09-29',
 'seats_number': 5,
 'seller_name': None,
 'title': 'CITROËN C3 BlueHDi 75KW (100CV) S&S Feel',
 'transmission': 'manual',
 'trim': 'BlueHDi 75KW (100CV) S&S Feel',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789056%27 HTTP/1.1" 200 None
2025-12-05 20:03:15 [ayvens.es] INFO: Saving data for 1789056: {'created_time': 1764964995.50459, 'last_price_update_time': 1764964995.504601, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789056') HTTP/1.1" 204 0
2025-12-05 20:03:15 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/104/1275ac45aac54053bcfea51f6dacd02e/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:15 [ayvens.es] INFO: Found listing with ID: 1789065
2025-12-05 20:03:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789065%27 HTTP/1.1" 200 None
2025-12-05 20:03:15 [ayvens.es] INFO: Scrape type for 1789065: 1
2025-12-05 20:03:15 [ayvens.es] INFO: Extracting data for car ID: 1789065
2025-12-05 20:03:15 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 1d850c8d-0af9-59da-827f-d1e55f71e35b with identifier: 1789065
2025-12-05 20:03:15 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 1d850c8d-0af9-59da-827f-d1e55f71e35b with scrape type 1
2025-12-05 20:03:15 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 1d850c8d-0af9-59da-827f-d1e55f71e35b
2025-12-05 20:03:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F329%252F697dceb32aaf4a7e8e00622a5b7ed6fe%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:15 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/329/697dceb32aaf4a7e8e00622a5b7ed6fe/images/large/1.jpg not downloaded yet
2025-12-05 20:03:15 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/677/1e76ebcdd95545e1a9eddbbc61b97138/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:15 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:15 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:15 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:15 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0438510>
2025-12-05 20:03:15 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:15 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:15 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:03:15 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:15 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:15 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:16 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_52758577-3d85-5b21-b21e-31b549b36868/52758577-3d85-5b21-b21e-31b549b36868_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F647%252F53486f6a82214db4a72d15579ea5167a%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:16 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/647/53486f6a82214db4a72d15579ea5167a/images/large/1.jpg saved to cache
2025-12-05 20:03:16 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 52758577-3d85-5b21-b21e-31b549b36868
2025-12-05 20:03:16 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 52758577-3d85-5b21-b21e-31b549b36868, skipping ID generation
2025-12-05 20:03:16 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 52758577-3d85-5b21-b21e-31b549b36868 with scrape type 1
2025-12-05 20:03:16 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 52758577-3d85-5b21-b21e-31b549b36868 from tracking dictionaries
2025-12-05 20:03:16 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 52758577-3d85-5b21-b21e-31b549b36868 sending to next pipeline
2025-12-05 20:03:16 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 52758577-3d85-5b21-b21e-31b549b36868, identifier: 1789057
2025-12-05 20:03:16 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:16 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:16 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:16 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0478150>
2025-12-05 20:03:16 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:16 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:16 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:03:16 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:16 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:16 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:16 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_8a040621-320e-5d5e-9e85-3d98feb9cdee/8a040621-320e-5d5e-9e85-3d98feb9cdee_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F647%252F5aa5e9c6b96248aca99f50ee960c46df%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:16 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/647/5aa5e9c6b96248aca99f50ee960c46df/images/large/1.jpg saved to cache
2025-12-05 20:03:16 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 8a040621-320e-5d5e-9e85-3d98feb9cdee
2025-12-05 20:03:16 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 8a040621-320e-5d5e-9e85-3d98feb9cdee, skipping ID generation
2025-12-05 20:03:16 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 8a040621-320e-5d5e-9e85-3d98feb9cdee with scrape type 1
2025-12-05 20:03:16 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 8a040621-320e-5d5e-9e85-3d98feb9cdee from tracking dictionaries
2025-12-05 20:03:16 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 8a040621-320e-5d5e-9e85-3d98feb9cdee sending to next pipeline
2025-12-05 20:03:16 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 8a040621-320e-5d5e-9e85-3d98feb9cdee, identifier: 1789058
2025-12-05 20:03:16 [ayvens.es] INFO: Found listing with ID: 1789066
2025-12-05 20:03:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789066%27 HTTP/1.1" 200 None
2025-12-05 20:03:16 [ayvens.es] INFO: Scrape type for 1789066: 1
2025-12-05 20:03:16 [ayvens.es] INFO: Extracting data for car ID: 1789066
2025-12-05 20:03:16 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: dec046fc-e57a-54da-bb98-b10b88823768 with identifier: 1789066
2025-12-05 20:03:16 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item dec046fc-e57a-54da-bb98-b10b88823768 with scrape type 1
2025-12-05 20:03:16 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item dec046fc-e57a-54da-bb98-b10b88823768
2025-12-05 20:03:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F676%252F64bcd436879f4122b4b7bf9359533536%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:16 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/676/64bcd436879f4122b4b7bf9359533536/images/large/1.jpg not downloaded yet
2025-12-05 20:03:16 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 52758577-3d85-5b21-b21e-31b549b36868 with identifier: 1789057
2025-12-05 20:03:16 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 52758577-3d85-5b21-b21e-31b549b36868 with identifier: 1789057
2025-12-05 20:03:16 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 52758577-3d85-5b21-b21e-31b549b36868 with identifier: 1789057 to the API
2025-12-05 20:03:16 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:16 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:16 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/647/53486f6a82214db4a72d15579ea5167a/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 125,
 'color': 'Negro',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '52758577-3d85-5b21-b21e-31b549b36868',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'KIA',
 'model': 'STONIC',
 'odometer': 44863,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_52758577-3d85-5b21-b21e-31b549b36868/52758577-3d85-5b21-b21e-31b549b36868_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 381778,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/647/53486f6a82214db4a72d15579ea5167a/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-07-23',
 'seats_number': 5,
 'seller_name': None,
 'title': 'KIA STONIC (O) 1.0 T-GDi 88kW (120CV) MHEV Drive DC',
 'transmission': 'manual',
 'trim': '(O) 1.0 T-GDi 88kW (120CV) MHEV Drive DC',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789057%27 HTTP/1.1" 200 None
2025-12-05 20:03:16 [ayvens.es] INFO: Saving data for 1789057: {'created_time': 1764964996.988479, 'last_price_update_time': 1764964996.988502, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789057') HTTP/1.1" 204 0
2025-12-05 20:03:17 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 8a040621-320e-5d5e-9e85-3d98feb9cdee with identifier: 1789058
2025-12-05 20:03:17 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 8a040621-320e-5d5e-9e85-3d98feb9cdee with identifier: 1789058
2025-12-05 20:03:17 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 8a040621-320e-5d5e-9e85-3d98feb9cdee with identifier: 1789058 to the API
2025-12-05 20:03:17 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:17 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:17 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/647/5aa5e9c6b96248aca99f50ee960c46df/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 125,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '8a040621-320e-5d5e-9e85-3d98feb9cdee',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'KIA',
 'model': 'STONIC',
 'odometer': 52935,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_8a040621-320e-5d5e-9e85-3d98feb9cdee/8a040621-320e-5d5e-9e85-3d98feb9cdee_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 355930,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/647/5aa5e9c6b96248aca99f50ee960c46df/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-10-13',
 'seats_number': 5,
 'seller_name': None,
 'title': 'KIA STONIC 1.0 T-GDi 88kW (120CV) MHEV Drive DCT',
 'transmission': 'manual',
 'trim': '1.0 T-GDi 88kW (120CV) MHEV Drive DCT',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789058%27 HTTP/1.1" 200 None
2025-12-05 20:03:17 [ayvens.es] INFO: Saving data for 1789058: {'created_time': 1764964997.36292, 'last_price_update_time': 1764964997.362929, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789058') HTTP/1.1" 204 0
2025-12-05 20:03:17 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/294/c4127973473c4b708d117cb6e72318c4/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:17 [ayvens.es] INFO: Found listing with ID: 1789067
2025-12-05 20:03:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789067%27 HTTP/1.1" 200 None
2025-12-05 20:03:17 [ayvens.es] INFO: Scrape type for 1789067: 1
2025-12-05 20:03:17 [ayvens.es] INFO: Extracting data for car ID: 1789067
2025-12-05 20:03:17 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: ba42179a-f52a-5eee-b524-ba688c0f81bb with identifier: 1789067
2025-12-05 20:03:17 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ba42179a-f52a-5eee-b524-ba688c0f81bb with scrape type 1
2025-12-05 20:03:17 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item ba42179a-f52a-5eee-b524-ba688c0f81bb
2025-12-05 20:03:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F255%252F1c48fdfafa15457c8d6a0d6046bb977b%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:17 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/255/1c48fdfafa15457c8d6a0d6046bb977b/images/large/1.jpg not downloaded yet
2025-12-05 20:03:17 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:17 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:17 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:17 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0521ed0>
2025-12-05 20:03:17 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:17 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:17 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:03:17 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:17 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_3cf4cdd2-94e2-5487-9768-c4b9b4fa5408/3cf4cdd2-94e2-5487-9768-c4b9b4fa5408_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F104%252F1275ac45aac54053bcfea51f6dacd02e%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:17 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/104/1275ac45aac54053bcfea51f6dacd02e/images/large/1.jpg saved to cache
2025-12-05 20:03:17 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 3cf4cdd2-94e2-5487-9768-c4b9b4fa5408
2025-12-05 20:03:17 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 3cf4cdd2-94e2-5487-9768-c4b9b4fa5408, skipping ID generation
2025-12-05 20:03:17 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 3cf4cdd2-94e2-5487-9768-c4b9b4fa5408 with scrape type 1
2025-12-05 20:03:17 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 3cf4cdd2-94e2-5487-9768-c4b9b4fa5408 from tracking dictionaries
2025-12-05 20:03:17 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 3cf4cdd2-94e2-5487-9768-c4b9b4fa5408 sending to next pipeline
2025-12-05 20:03:17 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 3cf4cdd2-94e2-5487-9768-c4b9b4fa5408, identifier: 1789059
2025-12-05 20:03:17 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:17 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:17 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:17 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b047b8d0>
2025-12-05 20:03:17 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:17 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:18 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1971
2025-12-05 20:03:18 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:18 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:18 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:18 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_485aee94-af0d-5e1c-886e-304a791500a4/485aee94-af0d-5e1c-886e-304a791500a4_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F677%252F1e76ebcdd95545e1a9eddbbc61b97138%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/677/1e76ebcdd95545e1a9eddbbc61b97138/images/large/1.jpg saved to cache
2025-12-05 20:03:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 485aee94-af0d-5e1c-886e-304a791500a4
2025-12-05 20:03:18 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 485aee94-af0d-5e1c-886e-304a791500a4, skipping ID generation
2025-12-05 20:03:18 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 485aee94-af0d-5e1c-886e-304a791500a4 with scrape type 1
2025-12-05 20:03:18 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 485aee94-af0d-5e1c-886e-304a791500a4 from tracking dictionaries
2025-12-05 20:03:18 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 485aee94-af0d-5e1c-886e-304a791500a4 sending to next pipeline
2025-12-05 20:03:18 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 485aee94-af0d-5e1c-886e-304a791500a4, identifier: 1789060
2025-12-05 20:03:18 [ayvens.es] INFO: Found listing with ID: 1789068
2025-12-05 20:03:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789068%27 HTTP/1.1" 200 None
2025-12-05 20:03:18 [ayvens.es] INFO: Scrape type for 1789068: 1
2025-12-05 20:03:18 [ayvens.es] INFO: Extracting data for car ID: 1789068
2025-12-05 20:03:18 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 70d855cf-806e-5979-be34-4401e9bb77c5 with identifier: 1789068
2025-12-05 20:03:18 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 70d855cf-806e-5979-be34-4401e9bb77c5 with scrape type 1
2025-12-05 20:03:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 70d855cf-806e-5979-be34-4401e9bb77c5
2025-12-05 20:03:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F270%252F52a517dbac8c4f56bacc4446e4cffdaa%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/270/52a517dbac8c4f56bacc4446e4cffdaa/images/large/1.jpg not downloaded yet
2025-12-05 20:03:18 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 3cf4cdd2-94e2-5487-9768-c4b9b4fa5408 with identifier: 1789059
2025-12-05 20:03:18 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:03:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:03:18 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 3cf4cdd2-94e2-5487-9768-c4b9b4fa5408 with identifier: 1789059
2025-12-05 20:03:18 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 3cf4cdd2-94e2-5487-9768-c4b9b4fa5408 with identifier: 1789059 to the API
2025-12-05 20:03:18 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:18 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:18 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/104/1275ac45aac54053bcfea51f6dacd02e/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 105,
 'color': 'Blanco metalizado',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '3cf4cdd2-94e2-5487-9768-c4b9b4fa5408',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'NISSAN',
 'model': 'MICRA',
 'odometer': 143425,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_3cf4cdd2-94e2-5487-9768-c4b9b4fa5408/3cf4cdd2-94e2-5487-9768-c4b9b4fa5408_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 342459,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/104/1275ac45aac54053bcfea51f6dacd02e/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-07-18',
 'seats_number': 5,
 'seller_name': None,
 'title': 'NISSAN MICRA IG-T 74 kW (100 CV) E6D Acenta',
 'transmission': 'manual',
 'trim': 'IG-T 74 kW (100 CV) E6D Acenta',
 'vin': None,
 'year': 2019}
2025-12-05 20:03:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789059%27 HTTP/1.1" 200 None
2025-12-05 20:03:18 [ayvens.es] INFO: Saving data for 1789059: {'created_time': 1764964998.69557, 'last_price_update_time': 1764964998.695581, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789059') HTTP/1.1" 204 0
2025-12-05 20:03:18 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 485aee94-af0d-5e1c-886e-304a791500a4 with identifier: 1789060
2025-12-05 20:03:18 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:03:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:03:18 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 485aee94-af0d-5e1c-886e-304a791500a4 with identifier: 1789060
2025-12-05 20:03:18 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'ford' containing 'ford' and model 'transit connect' containing 'transit'
2025-12-05 20:03:18 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 485aee94-af0d-5e1c-886e-304a791500a4 spider: ayvens.es
2025-12-05 20:03:18 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 485aee94-af0d-5e1c-886e-304a791500a4 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'ford' containing 'ford' and model 'transit connect' containing 'transit'
{'auction_title': '',
 'c_o_2_emission_value': 146,
 'color': 'Blanco s',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '485aee94-af0d-5e1c-886e-304a791500a4',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'FORD',
 'model': 'TRANSIT CONNECT',
 'odometer': 59851,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_485aee94-af0d-5e1c-886e-304a791500a4/485aee94-af0d-5e1c-886e-304a791500a4_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 335815,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/677/1e76ebcdd95545e1a9eddbbc61b97138/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2023-02-01',
 'seats_number': 2,
 'seller_name': None,
 'title': 'FORD TRANSIT CONNECT Van 1.5 TDCi 74kW Trend 200 L1',
 'transmission': 'manual',
 'trim': 'Van 1.5 TDCi 74kW Trend 200 L1',
 'vin': None,
 'year': 2023}
2025-12-05 20:03:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789060%27 HTTP/1.1" 200 None
2025-12-05 20:03:18 [ayvens.es] INFO: Saving data for 1789060: {'created_time': 1764964998.981586, 'last_price_update_time': 1764964998.981598, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789060') HTTP/1.1" 204 0
2025-12-05 20:03:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/407/7b3bff0f746d41d48db9cafa32ac96cd/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/770/4dc77b39377e4b21a9104bb7d9d93472/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:19 [ayvens.es] INFO: Found listing with ID: 1789069
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789069%27 HTTP/1.1" 200 None
2025-12-05 20:03:19 [ayvens.es] INFO: Scrape type for 1789069: 1
2025-12-05 20:03:19 [ayvens.es] INFO: Extracting data for car ID: 1789069
2025-12-05 20:03:19 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 7a1d9e19-fab5-5030-8398-ed6723946b33 with identifier: 1789069
2025-12-05 20:03:19 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 7a1d9e19-fab5-5030-8398-ed6723946b33 with scrape type 1
2025-12-05 20:03:19 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 7a1d9e19-fab5-5030-8398-ed6723946b33
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F358%252F0fdda9fb31974ab98027097dd46ed37c%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/358/0fdda9fb31974ab98027097dd46ed37c/images/large/1.jpg not downloaded yet
2025-12-05 20:03:19 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:19 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:19 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:19 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0adbd50>
2025-12-05 20:03:19 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:19 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_bfd49ef3-42d9-51aa-9a0b-c46efa552816/bfd49ef3-42d9-51aa-9a0b-c46efa552816_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F294%252Fc4127973473c4b708d117cb6e72318c4%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/294/c4127973473c4b708d117cb6e72318c4/images/large/1.jpg saved to cache
2025-12-05 20:03:19 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item bfd49ef3-42d9-51aa-9a0b-c46efa552816
2025-12-05 20:03:19 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: bfd49ef3-42d9-51aa-9a0b-c46efa552816, skipping ID generation
2025-12-05 20:03:19 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item bfd49ef3-42d9-51aa-9a0b-c46efa552816 with scrape type 1
2025-12-05 20:03:19 [crawlers.middlewares.photo_download_middleware] INFO: Removing item bfd49ef3-42d9-51aa-9a0b-c46efa552816 from tracking dictionaries
2025-12-05 20:03:19 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item bfd49ef3-42d9-51aa-9a0b-c46efa552816 sending to next pipeline
2025-12-05 20:03:19 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: bfd49ef3-42d9-51aa-9a0b-c46efa552816, identifier: 1789061
2025-12-05 20:03:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/329/697dceb32aaf4a7e8e00622a5b7ed6fe/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:19 [ayvens.es] INFO: Found listing with ID: 1789070
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789070%27 HTTP/1.1" 200 None
2025-12-05 20:03:19 [ayvens.es] INFO: Scrape type for 1789070: 1
2025-12-05 20:03:19 [ayvens.es] INFO: Extracting data for car ID: 1789070
2025-12-05 20:03:19 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 7518585d-badb-569c-8a68-0a8fb51034dc with identifier: 1789070
2025-12-05 20:03:19 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 7518585d-badb-569c-8a68-0a8fb51034dc with scrape type 1
2025-12-05 20:03:19 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 7518585d-badb-569c-8a68-0a8fb51034dc
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F559%252Fa967f2bde2c644a5bb8a03de9b4f0200%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/559/a967f2bde2c644a5bb8a03de9b4f0200/images/large/1.jpg not downloaded yet
2025-12-05 20:03:19 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: bfd49ef3-42d9-51aa-9a0b-c46efa552816 with identifier: 1789061
2025-12-05 20:03:19 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:03:19 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: bfd49ef3-42d9-51aa-9a0b-c46efa552816 with identifier: 1789061
2025-12-05 20:03:19 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'peugeot' containing 'peugeot' and model 'rifter' containing 'rifter'
2025-12-05 20:03:19 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: bfd49ef3-42d9-51aa-9a0b-c46efa552816 spider: ayvens.es
2025-12-05 20:03:19 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: bfd49ef3-42d9-51aa-9a0b-c46efa552816 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'peugeot' containing 'peugeot' and model 'rifter' containing 'rifter'
{'auction_title': '',
 'c_o_2_emission_value': 111,
 'color': 'Blanco banquise',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'bfd49ef3-42d9-51aa-9a0b-c46efa552816',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': 'RIFTER',
 'odometer': 138169,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_bfd49ef3-42d9-51aa-9a0b-c46efa552816/bfd49ef3-42d9-51aa-9a0b-c46efa552816_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 326560,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/294/c4127973473c4b708d117cb6e72318c4/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-03-27',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT RIFTER Access Standard BlueHDi 55kW',
 'transmission': 'manual',
 'trim': 'Access Standard BlueHDi 55kW',
 'vin': None,
 'year': 2019}
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789061%27 HTTP/1.1" 200 None
2025-12-05 20:03:19 [ayvens.es] INFO: Saving data for 1789061: {'created_time': 1764964999.814926, 'last_price_update_time': 1764964999.814938, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789061') HTTP/1.1" 204 0
2025-12-05 20:03:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/190/ac8f475ad9994eee80c955daa92bbaae/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:19 [ayvens.es] INFO: Found listing with ID: 1789071
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789071%27 HTTP/1.1" 200 None
2025-12-05 20:03:19 [ayvens.es] INFO: Scrape type for 1789071: 1
2025-12-05 20:03:19 [ayvens.es] INFO: Extracting data for car ID: 1789071
2025-12-05 20:03:19 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: e931264a-2306-5cf0-ae46-b7b71c8a6891 with identifier: 1789071
2025-12-05 20:03:19 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e931264a-2306-5cf0-ae46-b7b71c8a6891 with scrape type 1
2025-12-05 20:03:19 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item e931264a-2306-5cf0-ae46-b7b71c8a6891
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F169%252F0f5becf14a5e41c48907f405cb68ae30%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/169/0f5becf14a5e41c48907f405cb68ae30/images/large/1.jpg not downloaded yet
2025-12-05 20:03:19 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:19 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:19 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:19 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b91010>
2025-12-05 20:03:19 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:19 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:20 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:03:20 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:20 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:20 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:20 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_caeb8e5a-7ad9-5b8b-ae76-687a5404796b/caeb8e5a-7ad9-5b8b-ae76-687a5404796b_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F407%252F7b3bff0f746d41d48db9cafa32ac96cd%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/407/7b3bff0f746d41d48db9cafa32ac96cd/images/large/1.jpg saved to cache
2025-12-05 20:03:20 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item caeb8e5a-7ad9-5b8b-ae76-687a5404796b
2025-12-05 20:03:20 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: caeb8e5a-7ad9-5b8b-ae76-687a5404796b, skipping ID generation
2025-12-05 20:03:20 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item caeb8e5a-7ad9-5b8b-ae76-687a5404796b with scrape type 1
2025-12-05 20:03:20 [crawlers.middlewares.photo_download_middleware] INFO: Removing item caeb8e5a-7ad9-5b8b-ae76-687a5404796b from tracking dictionaries
2025-12-05 20:03:20 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item caeb8e5a-7ad9-5b8b-ae76-687a5404796b sending to next pipeline
2025-12-05 20:03:20 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: caeb8e5a-7ad9-5b8b-ae76-687a5404796b, identifier: 1789063
2025-12-05 20:03:20 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:20 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:20 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:20 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0408790>
2025-12-05 20:03:20 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:20 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:20 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:03:20 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:20 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:20 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:20 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_10e434db-19c6-5f46-b867-84c7c701fa24/10e434db-19c6-5f46-b867-84c7c701fa24_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F770%252F4dc77b39377e4b21a9104bb7d9d93472%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/770/4dc77b39377e4b21a9104bb7d9d93472/images/large/1.jpg saved to cache
2025-12-05 20:03:20 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 10e434db-19c6-5f46-b867-84c7c701fa24
2025-12-05 20:03:20 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 10e434db-19c6-5f46-b867-84c7c701fa24, skipping ID generation
2025-12-05 20:03:20 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 10e434db-19c6-5f46-b867-84c7c701fa24 with scrape type 1
2025-12-05 20:03:20 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 10e434db-19c6-5f46-b867-84c7c701fa24 from tracking dictionaries
2025-12-05 20:03:20 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 10e434db-19c6-5f46-b867-84c7c701fa24 sending to next pipeline
2025-12-05 20:03:20 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 10e434db-19c6-5f46-b867-84c7c701fa24, identifier: 1789062
2025-12-05 20:03:20 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:20 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:20 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:20 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b051fe10>
2025-12-05 20:03:20 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:20 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:20 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1971
2025-12-05 20:03:20 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:21 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:21 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:21 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_1d850c8d-0af9-59da-827f-d1e55f71e35b/1d850c8d-0af9-59da-827f-d1e55f71e35b_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F329%252F697dceb32aaf4a7e8e00622a5b7ed6fe%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/329/697dceb32aaf4a7e8e00622a5b7ed6fe/images/large/1.jpg saved to cache
2025-12-05 20:03:21 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 1d850c8d-0af9-59da-827f-d1e55f71e35b
2025-12-05 20:03:21 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 1d850c8d-0af9-59da-827f-d1e55f71e35b, skipping ID generation
2025-12-05 20:03:21 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 1d850c8d-0af9-59da-827f-d1e55f71e35b with scrape type 1
2025-12-05 20:03:21 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 1d850c8d-0af9-59da-827f-d1e55f71e35b from tracking dictionaries
2025-12-05 20:03:21 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 1d850c8d-0af9-59da-827f-d1e55f71e35b sending to next pipeline
2025-12-05 20:03:21 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 1d850c8d-0af9-59da-827f-d1e55f71e35b, identifier: 1789065
2025-12-05 20:03:21 [ayvens.es] INFO: Found listing with ID: 1789072
2025-12-05 20:03:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789072%27 HTTP/1.1" 200 None
2025-12-05 20:03:21 [ayvens.es] INFO: Scrape type for 1789072: 1
2025-12-05 20:03:21 [ayvens.es] INFO: Extracting data for car ID: 1789072
2025-12-05 20:03:21 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 8c2945fb-92ba-50e9-803c-41788dfb81cf with identifier: 1789072
2025-12-05 20:03:21 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 8c2945fb-92ba-50e9-803c-41788dfb81cf with scrape type 1
2025-12-05 20:03:21 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 8c2945fb-92ba-50e9-803c-41788dfb81cf
2025-12-05 20:03:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F086%252F07b2b1b1487e4beb90577a1ecf46e86a%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/086/07b2b1b1487e4beb90577a1ecf46e86a/images/large/1.jpg not downloaded yet
2025-12-05 20:03:21 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: caeb8e5a-7ad9-5b8b-ae76-687a5404796b with identifier: 1789063
2025-12-05 20:03:21 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: caeb8e5a-7ad9-5b8b-ae76-687a5404796b with identifier: 1789063
2025-12-05 20:03:21 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'renault' containing 'renault' and model 'kangoo' containing 'kangoo'
2025-12-05 20:03:21 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: caeb8e5a-7ad9-5b8b-ae76-687a5404796b spider: ayvens.es
2025-12-05 20:03:21 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: caeb8e5a-7ad9-5b8b-ae76-687a5404796b due to rule: makes_models_not_allowed.json, error: Rejected due to make 'renault' containing 'renault' and model 'kangoo' containing 'kangoo'
{'auction_title': '',
 'c_o_2_emission_value': 123,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'caeb8e5a-7ad9-5b8b-ae76-687a5404796b',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'KANGOO',
 'odometer': 145049,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_caeb8e5a-7ad9-5b8b-ae76-687a5404796b/caeb8e5a-7ad9-5b8b-ae76-687a5404796b_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 341246,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/407/7b3bff0f746d41d48db9cafa32ac96cd/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2018-04-13',
 'seats_number': 5,
 'seller_name': None,
 'title': 'RENAULT KANGOO Profesional N1 Energy dCi 66kW (90CV) E6',
 'transmission': 'manual',
 'trim': 'Profesional N1 Energy dCi 66kW (90CV) E6',
 'vin': None,
 'year': 2018}
2025-12-05 20:03:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789063%27 HTTP/1.1" 200 None
2025-12-05 20:03:21 [ayvens.es] INFO: Saving data for 1789063: {'created_time': 1764965001.592554, 'last_price_update_time': 1764965001.592566, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789063') HTTP/1.1" 204 0
2025-12-05 20:03:21 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 10e434db-19c6-5f46-b867-84c7c701fa24 with identifier: 1789062
2025-12-05 20:03:21 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 10e434db-19c6-5f46-b867-84c7c701fa24 with identifier: 1789062
2025-12-05 20:03:21 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'renault' containing 'renault' and model 'trafic' containing 'trafic'
2025-12-05 20:03:21 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 10e434db-19c6-5f46-b867-84c7c701fa24 spider: ayvens.es
2025-12-05 20:03:21 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 10e434db-19c6-5f46-b867-84c7c701fa24 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'renault' containing 'renault' and model 'trafic' containing 'trafic'
{'auction_title': '',
 'c_o_2_emission_value': 182,
 'color': 'Serie',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '10e434db-19c6-5f46-b867-84c7c701fa24',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'TRAFIC',
 'odometer': 198958,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_10e434db-19c6-5f46-b867-84c7c701fa24/10e434db-19c6-5f46-b867-84c7c701fa24_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 344369,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/770/4dc77b39377e4b21a9104bb7d9d93472/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2022-05-30',
 'seats_number': 3,
 'seller_name': None,
 'title': 'RENAULT TRAFIC Furgon L1H1 Blue dCi 96 Kw (130 CV)',
 'transmission': 'manual',
 'trim': 'Furgon L1H1 Blue dCi 96 Kw (130 CV)',
 'vin': None,
 'year': 2022}
2025-12-05 20:03:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789062%27 HTTP/1.1" 200 None
2025-12-05 20:03:21 [ayvens.es] INFO: Saving data for 1789062: {'created_time': 1764965001.932764, 'last_price_update_time': 1764965001.932773, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789062') HTTP/1.1" 204 0
2025-12-05 20:03:21 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 1d850c8d-0af9-59da-827f-d1e55f71e35b with identifier: 1789065
2025-12-05 20:03:21 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 1d850c8d-0af9-59da-827f-d1e55f71e35b with identifier: 1789065
2025-12-05 20:03:22 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 1d850c8d-0af9-59da-827f-d1e55f71e35b with identifier: 1789065 to the API
2025-12-05 20:03:22 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:22 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:22 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/329/697dceb32aaf4a7e8e00622a5b7ed6fe/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 148,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '1d850c8d-0af9-59da-827f-d1e55f71e35b',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'SKODA',
 'model': 'KAROQ',
 'odometer': 148523,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_1d850c8d-0af9-59da-827f-d1e55f71e35b/1d850c8d-0af9-59da-827f-d1e55f71e35b_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 314985,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/329/697dceb32aaf4a7e8e00622a5b7ed6fe/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-03-18',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SKODA KAROQ 2.0 TDI 110kW (150CV) DSG 4X4 Ambition',
 'transmission': 'manual',
 'trim': '2.0 TDI 110kW (150CV) DSG 4X4 Ambition',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789065%27 HTTP/1.1" 200 None
2025-12-05 20:03:22 [ayvens.es] INFO: Saving data for 1789065: {'created_time': 1764965002.331887, 'last_price_update_time': 1764965002.331897, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789065') HTTP/1.1" 204 0
2025-12-05 20:03:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/255/1c48fdfafa15457c8d6a0d6046bb977b/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/676/64bcd436879f4122b4b7bf9359533536/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:22 [ayvens.es] INFO: Found listing with ID: 1789073
2025-12-05 20:03:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789073%27 HTTP/1.1" 200 None
2025-12-05 20:03:22 [ayvens.es] INFO: Scrape type for 1789073: 1
2025-12-05 20:03:22 [ayvens.es] INFO: Extracting data for car ID: 1789073
2025-12-05 20:03:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: c8d8caba-95dd-5021-b550-d4c90dc918fe with identifier: 1789073
2025-12-05 20:03:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item c8d8caba-95dd-5021-b550-d4c90dc918fe with scrape type 1
2025-12-05 20:03:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item c8d8caba-95dd-5021-b550-d4c90dc918fe
2025-12-05 20:03:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F636%252F640b2208cc3442c5b727c20104939888%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/636/640b2208cc3442c5b727c20104939888/images/large/1.jpg not downloaded yet
2025-12-05 20:03:22 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:22 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:22 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:22 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0699850>
2025-12-05 20:03:22 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:22 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:22 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:03:22 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:22 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:22 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:22 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_8a0c3831-cf50-5b2c-b2fe-6af14adb4f07/8a0c3831-cf50-5b2c-b2fe-6af14adb4f07_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F190%252Fac8f475ad9994eee80c955daa92bbaae%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/190/ac8f475ad9994eee80c955daa92bbaae/images/large/1.jpg saved to cache
2025-12-05 20:03:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 8a0c3831-cf50-5b2c-b2fe-6af14adb4f07
2025-12-05 20:03:22 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 8a0c3831-cf50-5b2c-b2fe-6af14adb4f07, skipping ID generation
2025-12-05 20:03:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 8a0c3831-cf50-5b2c-b2fe-6af14adb4f07 with scrape type 1
2025-12-05 20:03:22 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 8a0c3831-cf50-5b2c-b2fe-6af14adb4f07 from tracking dictionaries
2025-12-05 20:03:22 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 8a0c3831-cf50-5b2c-b2fe-6af14adb4f07 sending to next pipeline
2025-12-05 20:03:22 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 8a0c3831-cf50-5b2c-b2fe-6af14adb4f07, identifier: 1789064
2025-12-05 20:03:22 [ayvens.es] INFO: Found listing with ID: 1789074
2025-12-05 20:03:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789074%27 HTTP/1.1" 200 None
2025-12-05 20:03:22 [ayvens.es] INFO: Scrape type for 1789074: 1
2025-12-05 20:03:22 [ayvens.es] INFO: Extracting data for car ID: 1789074
2025-12-05 20:03:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: f400ff90-c505-594b-b099-784624b1e330 with identifier: 1789074
2025-12-05 20:03:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f400ff90-c505-594b-b099-784624b1e330 with scrape type 1
2025-12-05 20:03:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item f400ff90-c505-594b-b099-784624b1e330
2025-12-05 20:03:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F597%252F9d095159304349c2bb2706484298cbb8%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/597/9d095159304349c2bb2706484298cbb8/images/large/1.jpg not downloaded yet
2025-12-05 20:03:22 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 8a0c3831-cf50-5b2c-b2fe-6af14adb4f07 with identifier: 1789064
2025-12-05 20:03:22 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 8a0c3831-cf50-5b2c-b2fe-6af14adb4f07 with identifier: 1789064
2025-12-05 20:03:23 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'opel' containing 'opel' and model 'combo' containing 'combo'
2025-12-05 20:03:23 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 8a0c3831-cf50-5b2c-b2fe-6af14adb4f07 spider: ayvens.es
2025-12-05 20:03:23 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 8a0c3831-cf50-5b2c-b2fe-6af14adb4f07 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'opel' containing 'opel' and model 'combo' containing 'combo'
{'auction_title': '',
 'c_o_2_emission_value': 133,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 3,
 'engine': '1.2L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '8a0c3831-cf50-5b2c-b2fe-6af14adb4f07',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'OPEL',
 'model': 'COMBO',
 'odometer': 150003,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_8a0c3831-cf50-5b2c-b2fe-6af14adb4f07/8a0c3831-cf50-5b2c-b2fe-6af14adb4f07_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 322517,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/190/ac8f475ad9994eee80c955daa92bbaae/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2017-09-22',
 'seats_number': 2,
 'seller_name': None,
 'title': 'OPEL COMBO Cargo 1.3 CDTI 95CV L1 H1',
 'transmission': 'manual',
 'trim': 'Cargo 1.3 CDTI 95CV L1 H1',
 'vin': None,
 'year': 2017}
2025-12-05 20:03:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789064%27 HTTP/1.1" 200 None
2025-12-05 20:03:23 [ayvens.es] INFO: Saving data for 1789064: {'created_time': 1764965003.225989, 'last_price_update_time': 1764965003.226001, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789064') HTTP/1.1" 204 0
2025-12-05 20:03:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/270/52a517dbac8c4f56bacc4446e4cffdaa/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/358/0fdda9fb31974ab98027097dd46ed37c/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:23 [ayvens.es] INFO: Found listing with ID: 1789075
2025-12-05 20:03:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789075%27 HTTP/1.1" 200 None
2025-12-05 20:03:23 [ayvens.es] INFO: Scrape type for 1789075: 1
2025-12-05 20:03:23 [ayvens.es] INFO: Extracting data for car ID: 1789075
2025-12-05 20:03:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: cad47282-42ae-5da8-9918-9a44e04455b1 with identifier: 1789075
2025-12-05 20:03:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item cad47282-42ae-5da8-9918-9a44e04455b1 with scrape type 1
2025-12-05 20:03:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item cad47282-42ae-5da8-9918-9a44e04455b1
2025-12-05 20:03:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F409%252F5fc5797b95af4219a6ced3d5e6ab54f6%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/409/5fc5797b95af4219a6ced3d5e6ab54f6/images/large/1.jpg not downloaded yet
2025-12-05 20:03:23 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:23 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:23 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:23 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0503990>
2025-12-05 20:03:23 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:23 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:23 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:03:23 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:23 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:23 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:23 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_ba42179a-f52a-5eee-b524-ba688c0f81bb/ba42179a-f52a-5eee-b524-ba688c0f81bb_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F255%252F1c48fdfafa15457c8d6a0d6046bb977b%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/255/1c48fdfafa15457c8d6a0d6046bb977b/images/large/1.jpg saved to cache
2025-12-05 20:03:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item ba42179a-f52a-5eee-b524-ba688c0f81bb
2025-12-05 20:03:23 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: ba42179a-f52a-5eee-b524-ba688c0f81bb, skipping ID generation
2025-12-05 20:03:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ba42179a-f52a-5eee-b524-ba688c0f81bb with scrape type 1
2025-12-05 20:03:23 [crawlers.middlewares.photo_download_middleware] INFO: Removing item ba42179a-f52a-5eee-b524-ba688c0f81bb from tracking dictionaries
2025-12-05 20:03:23 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item ba42179a-f52a-5eee-b524-ba688c0f81bb sending to next pipeline
2025-12-05 20:03:23 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: ba42179a-f52a-5eee-b524-ba688c0f81bb, identifier: 1789067
2025-12-05 20:03:23 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:23 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:23 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:23 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0dc2050>
2025-12-05 20:03:23 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:23 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:23 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:03:23 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:24 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:24 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:24 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_dec046fc-e57a-54da-bb98-b10b88823768/dec046fc-e57a-54da-bb98-b10b88823768_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F676%252F64bcd436879f4122b4b7bf9359533536%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/676/64bcd436879f4122b4b7bf9359533536/images/large/1.jpg saved to cache
2025-12-05 20:03:24 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item dec046fc-e57a-54da-bb98-b10b88823768
2025-12-05 20:03:24 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: dec046fc-e57a-54da-bb98-b10b88823768, skipping ID generation
2025-12-05 20:03:24 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item dec046fc-e57a-54da-bb98-b10b88823768 with scrape type 1
2025-12-05 20:03:24 [crawlers.middlewares.photo_download_middleware] INFO: Removing item dec046fc-e57a-54da-bb98-b10b88823768 from tracking dictionaries
2025-12-05 20:03:24 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item dec046fc-e57a-54da-bb98-b10b88823768 sending to next pipeline
2025-12-05 20:03:24 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: dec046fc-e57a-54da-bb98-b10b88823768, identifier: 1789066
2025-12-05 20:03:24 [ayvens.es] INFO: Found listing with ID: 1789076
2025-12-05 20:03:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789076%27 HTTP/1.1" 200 None
2025-12-05 20:03:24 [ayvens.es] INFO: Scrape type for 1789076: 1
2025-12-05 20:03:24 [ayvens.es] INFO: Extracting data for car ID: 1789076
2025-12-05 20:03:24 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 882ff929-ae62-5f3d-96e9-cdbb5434aff8 with identifier: 1789076
2025-12-05 20:03:24 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 882ff929-ae62-5f3d-96e9-cdbb5434aff8 with scrape type 1
2025-12-05 20:03:24 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 882ff929-ae62-5f3d-96e9-cdbb5434aff8
2025-12-05 20:03:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F324%252F598c2a198ab5472f97322eef01f70751%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/324/598c2a198ab5472f97322eef01f70751/images/large/1.jpg not downloaded yet
2025-12-05 20:03:24 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: ba42179a-f52a-5eee-b524-ba688c0f81bb with identifier: 1789067
2025-12-05 20:03:24 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: ba42179a-f52a-5eee-b524-ba688c0f81bb with identifier: 1789067
2025-12-05 20:03:24 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'renault' containing 'renault' and model 'kangoo' containing 'kangoo'
2025-12-05 20:03:24 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: ba42179a-f52a-5eee-b524-ba688c0f81bb spider: ayvens.es
2025-12-05 20:03:24 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: ba42179a-f52a-5eee-b524-ba688c0f81bb due to rule: makes_models_not_allowed.json, error: Rejected due to make 'renault' containing 'renault' and model 'kangoo' containing 'kangoo'
{'auction_title': '',
 'c_o_2_emission_value': 123,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'ba42179a-f52a-5eee-b524-ba688c0f81bb',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'KANGOO',
 'odometer': 195144,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ba42179a-f52a-5eee-b524-ba688c0f81bb/ba42179a-f52a-5eee-b524-ba688c0f81bb_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 337361,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/255/1c48fdfafa15457c8d6a0d6046bb977b/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-03-22',
 'seats_number': 5,
 'seller_name': None,
 'title': 'RENAULT KANGOO Profesional N1 Energy dCi 66kW (90CV)',
 'transmission': 'manual',
 'trim': 'Profesional N1 Energy dCi 66kW (90CV)',
 'vin': None,
 'year': 2019}
2025-12-05 20:03:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789067%27 HTTP/1.1" 200 None
2025-12-05 20:03:24 [ayvens.es] INFO: Saving data for 1789067: {'created_time': 1764965004.529524, 'last_price_update_time': 1764965004.529535, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789067') HTTP/1.1" 204 0
2025-12-05 20:03:24 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: dec046fc-e57a-54da-bb98-b10b88823768 with identifier: 1789066
2025-12-05 20:03:24 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: dec046fc-e57a-54da-bb98-b10b88823768 with identifier: 1789066
2025-12-05 20:03:24 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'renault' containing 'renault' and model 'kangoo' containing 'kangoo'
2025-12-05 20:03:24 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: dec046fc-e57a-54da-bb98-b10b88823768 spider: ayvens.es
2025-12-05 20:03:24 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: dec046fc-e57a-54da-bb98-b10b88823768 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'renault' containing 'renault' and model 'kangoo' containing 'kangoo'
{'auction_title': '',
 'c_o_2_emission_value': 130,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'dec046fc-e57a-54da-bb98-b10b88823768',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'KANGOO',
 'odometer': 170079,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_dec046fc-e57a-54da-bb98-b10b88823768/dec046fc-e57a-54da-bb98-b10b88823768_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 333012,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/676/64bcd436879f4122b4b7bf9359533536/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2016-05-18',
 'seats_number': 5,
 'seller_name': None,
 'title': 'RENAULT KANGOO (E) Profesional N1 dCi 75',
 'transmission': 'manual',
 'trim': '(E) Profesional N1 dCi 75',
 'vin': None,
 'year': 2016}
2025-12-05 20:03:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789066%27 HTTP/1.1" 200 None
2025-12-05 20:03:24 [ayvens.es] INFO: Saving data for 1789066: {'created_time': 1764965004.798271, 'last_price_update_time': 1764965004.798282, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789066') HTTP/1.1" 204 0
2025-12-05 20:03:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/559/a967f2bde2c644a5bb8a03de9b4f0200/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/169/0f5becf14a5e41c48907f405cb68ae30/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:24 [ayvens.es] INFO: Found listing with ID: 1789077
2025-12-05 20:03:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789077%27 HTTP/1.1" 200 None
2025-12-05 20:03:24 [ayvens.es] INFO: Scrape type for 1789077: 1
2025-12-05 20:03:24 [ayvens.es] INFO: Extracting data for car ID: 1789077
2025-12-05 20:03:24 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 96c9774f-52bd-5ee7-b544-64b990a7a20b with identifier: 1789077
2025-12-05 20:03:24 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 96c9774f-52bd-5ee7-b544-64b990a7a20b with scrape type 1
2025-12-05 20:03:24 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 96c9774f-52bd-5ee7-b544-64b990a7a20b
2025-12-05 20:03:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F983%252Fa4f5444e64d34dd7b7663132d8f65a13%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/983/a4f5444e64d34dd7b7663132d8f65a13/images/large/1.jpg not downloaded yet
2025-12-05 20:03:24 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:24 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:24 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:24 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b40810>
2025-12-05 20:03:24 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:24 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:25 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:03:25 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:25 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:25 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:25 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_70d855cf-806e-5979-be34-4401e9bb77c5/70d855cf-806e-5979-be34-4401e9bb77c5_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F270%252F52a517dbac8c4f56bacc4446e4cffdaa%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/270/52a517dbac8c4f56bacc4446e4cffdaa/images/large/1.jpg saved to cache
2025-12-05 20:03:25 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 70d855cf-806e-5979-be34-4401e9bb77c5
2025-12-05 20:03:25 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 70d855cf-806e-5979-be34-4401e9bb77c5, skipping ID generation
2025-12-05 20:03:25 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 70d855cf-806e-5979-be34-4401e9bb77c5 with scrape type 1
2025-12-05 20:03:25 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 70d855cf-806e-5979-be34-4401e9bb77c5 from tracking dictionaries
2025-12-05 20:03:25 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 70d855cf-806e-5979-be34-4401e9bb77c5 sending to next pipeline
2025-12-05 20:03:25 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 70d855cf-806e-5979-be34-4401e9bb77c5, identifier: 1789068
2025-12-05 20:03:25 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:25 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:25 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:25 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0479850>
2025-12-05 20:03:25 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:25 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:25 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:03:25 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:25 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:25 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:25 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_7a1d9e19-fab5-5030-8398-ed6723946b33/7a1d9e19-fab5-5030-8398-ed6723946b33_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F358%252F0fdda9fb31974ab98027097dd46ed37c%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/358/0fdda9fb31974ab98027097dd46ed37c/images/large/1.jpg saved to cache
2025-12-05 20:03:25 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 7a1d9e19-fab5-5030-8398-ed6723946b33
2025-12-05 20:03:25 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 7a1d9e19-fab5-5030-8398-ed6723946b33, skipping ID generation
2025-12-05 20:03:25 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 7a1d9e19-fab5-5030-8398-ed6723946b33 with scrape type 1
2025-12-05 20:03:25 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 7a1d9e19-fab5-5030-8398-ed6723946b33 from tracking dictionaries
2025-12-05 20:03:25 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 7a1d9e19-fab5-5030-8398-ed6723946b33 sending to next pipeline
2025-12-05 20:03:25 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 7a1d9e19-fab5-5030-8398-ed6723946b33, identifier: 1789069
2025-12-05 20:03:25 [ayvens.es] INFO: Found listing with ID: 1789078
2025-12-05 20:03:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789078%27 HTTP/1.1" 200 None
2025-12-05 20:03:25 [ayvens.es] INFO: Scrape type for 1789078: 1
2025-12-05 20:03:25 [ayvens.es] INFO: Extracting data for car ID: 1789078
2025-12-05 20:03:25 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 30d120c0-92b9-55a4-8742-7deb38ed42d6 with identifier: 1789078
2025-12-05 20:03:25 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 30d120c0-92b9-55a4-8742-7deb38ed42d6 with scrape type 1
2025-12-05 20:03:25 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 30d120c0-92b9-55a4-8742-7deb38ed42d6
2025-12-05 20:03:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F194%252F6ec4aef21025451386604dc3673b1e0f%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/194/6ec4aef21025451386604dc3673b1e0f/images/large/1.jpg not downloaded yet
2025-12-05 20:03:25 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 70d855cf-806e-5979-be34-4401e9bb77c5 with identifier: 1789068
2025-12-05 20:03:25 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 70d855cf-806e-5979-be34-4401e9bb77c5 with identifier: 1789068
2025-12-05 20:03:26 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'volkswagen' containing 'volkswagen' and model 'caddy' containing 'caddy'
2025-12-05 20:03:26 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 70d855cf-806e-5979-be34-4401e9bb77c5 spider: ayvens.es
2025-12-05 20:03:26 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 70d855cf-806e-5979-be34-4401e9bb77c5 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'volkswagen' containing 'volkswagen' and model 'caddy' containing 'caddy'
{'auction_title': '',
 'c_o_2_emission_value': 143,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '70d855cf-806e-5979-be34-4401e9bb77c5',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'CADDY',
 'odometer': 150077,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_70d855cf-806e-5979-be34-4401e9bb77c5/70d855cf-806e-5979-be34-4401e9bb77c5_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 337404,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/270/52a517dbac8c4f56bacc4446e4cffdaa/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2017-04-28',
 'seats_number': 5,
 'seller_name': None,
 'title': 'VOLKSWAGEN CADDY Profesional Kombi 2.0 TDI SCR 4M BMT 122',
 'transmission': 'manual',
 'trim': 'Profesional Kombi 2.0 TDI SCR 4M BMT 122',
 'vin': None,
 'year': 2017}
2025-12-05 20:03:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789068%27 HTTP/1.1" 200 None
2025-12-05 20:03:26 [ayvens.es] INFO: Saving data for 1789068: {'created_time': 1764965006.162338, 'last_price_update_time': 1764965006.16235, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789068') HTTP/1.1" 204 0
2025-12-05 20:03:26 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 7a1d9e19-fab5-5030-8398-ed6723946b33 with identifier: 1789069
2025-12-05 20:03:26 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 7a1d9e19-fab5-5030-8398-ed6723946b33 with identifier: 1789069
2025-12-05 20:03:26 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'opel' containing 'opel' and model 'combo' containing 'combo'
2025-12-05 20:03:26 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 7a1d9e19-fab5-5030-8398-ed6723946b33 spider: ayvens.es
2025-12-05 20:03:26 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 7a1d9e19-fab5-5030-8398-ed6723946b33 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'opel' containing 'opel' and model 'combo' containing 'combo'
{'auction_title': '',
 'c_o_2_emission_value': 132,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 3,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '7a1d9e19-fab5-5030-8398-ed6723946b33',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'OPEL',
 'model': 'COMBO',
 'odometer': 127884,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_7a1d9e19-fab5-5030-8398-ed6723946b33/7a1d9e19-fab5-5030-8398-ed6723946b33_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 339112,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/358/0fdda9fb31974ab98027097dd46ed37c/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-06-30',
 'seats_number': 2,
 'seller_name': None,
 'title': 'OPEL COMBO 1.5 TD 75kW (100CV) Express L H1 650kg',
 'transmission': 'manual',
 'trim': '1.5 TD 75kW (100CV) Express L H1 650kg',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789069%27 HTTP/1.1" 200 None
2025-12-05 20:03:26 [ayvens.es] INFO: Saving data for 1789069: {'created_time': 1764965006.415099, 'last_price_update_time': 1764965006.415112, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789069') HTTP/1.1" 204 0
2025-12-05 20:03:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/636/640b2208cc3442c5b727c20104939888/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/086/07b2b1b1487e4beb90577a1ecf46e86a/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:26 [ayvens.es] INFO: Found listing with ID: 1789079
2025-12-05 20:03:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789079%27 HTTP/1.1" 200 None
2025-12-05 20:03:26 [ayvens.es] INFO: Scrape type for 1789079: 1
2025-12-05 20:03:26 [ayvens.es] INFO: Extracting data for car ID: 1789079
2025-12-05 20:03:26 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 06d5c6d4-4b49-505c-bc95-20bd28d30341 with identifier: 1789079
2025-12-05 20:03:26 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 06d5c6d4-4b49-505c-bc95-20bd28d30341 with scrape type 1
2025-12-05 20:03:26 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 06d5c6d4-4b49-505c-bc95-20bd28d30341
2025-12-05 20:03:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F018%252Faa89fb84055a466c8b1feb6fad039b35%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/018/aa89fb84055a466c8b1feb6fad039b35/images/large/1.jpg not downloaded yet
2025-12-05 20:03:26 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:26 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:26 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:26 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b2a28d10>
2025-12-05 20:03:26 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:26 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:26 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:03:26 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:26 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:26 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:26 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_7518585d-badb-569c-8a68-0a8fb51034dc/7518585d-badb-569c-8a68-0a8fb51034dc_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F559%252Fa967f2bde2c644a5bb8a03de9b4f0200%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/559/a967f2bde2c644a5bb8a03de9b4f0200/images/large/1.jpg saved to cache
2025-12-05 20:03:26 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 7518585d-badb-569c-8a68-0a8fb51034dc
2025-12-05 20:03:26 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 7518585d-badb-569c-8a68-0a8fb51034dc, skipping ID generation
2025-12-05 20:03:26 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 7518585d-badb-569c-8a68-0a8fb51034dc with scrape type 1
2025-12-05 20:03:26 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 7518585d-badb-569c-8a68-0a8fb51034dc from tracking dictionaries
2025-12-05 20:03:26 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 7518585d-badb-569c-8a68-0a8fb51034dc sending to next pipeline
2025-12-05 20:03:26 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 7518585d-badb-569c-8a68-0a8fb51034dc, identifier: 1789070
2025-12-05 20:03:26 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:26 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:26 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:26 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0df36d0>
2025-12-05 20:03:26 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:26 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:27 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:03:27 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:27 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:27 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:27 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_e931264a-2306-5cf0-ae46-b7b71c8a6891/e931264a-2306-5cf0-ae46-b7b71c8a6891_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F169%252F0f5becf14a5e41c48907f405cb68ae30%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/169/0f5becf14a5e41c48907f405cb68ae30/images/large/1.jpg saved to cache
2025-12-05 20:03:27 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item e931264a-2306-5cf0-ae46-b7b71c8a6891
2025-12-05 20:03:27 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: e931264a-2306-5cf0-ae46-b7b71c8a6891, skipping ID generation
2025-12-05 20:03:27 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e931264a-2306-5cf0-ae46-b7b71c8a6891 with scrape type 1
2025-12-05 20:03:27 [crawlers.middlewares.photo_download_middleware] INFO: Removing item e931264a-2306-5cf0-ae46-b7b71c8a6891 from tracking dictionaries
2025-12-05 20:03:27 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item e931264a-2306-5cf0-ae46-b7b71c8a6891 sending to next pipeline
2025-12-05 20:03:27 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: e931264a-2306-5cf0-ae46-b7b71c8a6891, identifier: 1789071
2025-12-05 20:03:27 [ayvens.es] INFO: Fetching listings with token: 3b2cc62fd26c4e29a762db3de181266b and skip: 200
2025-12-05 20:03:27 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 7518585d-badb-569c-8a68-0a8fb51034dc with identifier: 1789070
2025-12-05 20:03:27 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 7518585d-badb-569c-8a68-0a8fb51034dc with identifier: 1789070
2025-12-05 20:03:27 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'renault' containing 'renault' and model 'kangoo' containing 'kangoo'
2025-12-05 20:03:27 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 7518585d-badb-569c-8a68-0a8fb51034dc spider: ayvens.es
2025-12-05 20:03:27 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 7518585d-badb-569c-8a68-0a8fb51034dc due to rule: makes_models_not_allowed.json, error: Rejected due to make 'renault' containing 'renault' and model 'kangoo' containing 'kangoo'
{'auction_title': '',
 'c_o_2_emission_value': 123,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '7518585d-badb-569c-8a68-0a8fb51034dc',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'KANGOO',
 'odometer': 129723,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_7518585d-badb-569c-8a68-0a8fb51034dc/7518585d-badb-569c-8a68-0a8fb51034dc_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 318890,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/559/a967f2bde2c644a5bb8a03de9b4f0200/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-04-12',
 'seats_number': 5,
 'seller_name': None,
 'title': 'RENAULT KANGOO Profesional N1 Energy dCi 66kW (90CV)',
 'transmission': 'manual',
 'trim': 'Profesional N1 Energy dCi 66kW (90CV)',
 'vin': None,
 'year': 2019}
2025-12-05 20:03:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789070%27 HTTP/1.1" 200 None
2025-12-05 20:03:27 [ayvens.es] INFO: Saving data for 1789070: {'created_time': 1764965007.625767, 'last_price_update_time': 1764965007.625778, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789070') HTTP/1.1" 204 0
2025-12-05 20:03:27 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: e931264a-2306-5cf0-ae46-b7b71c8a6891 with identifier: 1789071
2025-12-05 20:03:27 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:03:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:03:27 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: e931264a-2306-5cf0-ae46-b7b71c8a6891 with identifier: 1789071
2025-12-05 20:03:27 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'fiat' containing 'fiat' and model 'doblo cargo' containing 'doblo'
2025-12-05 20:03:27 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: e931264a-2306-5cf0-ae46-b7b71c8a6891 spider: ayvens.es
2025-12-05 20:03:27 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: e931264a-2306-5cf0-ae46-b7b71c8a6891 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'fiat' containing 'fiat' and model 'doblo cargo' containing 'doblo'
{'auction_title': '',
 'c_o_2_emission_value': 133,
 'color': 'Blanco gelato',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.2L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'e931264a-2306-5cf0-ae46-b7b71c8a6891',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'FIAT',
 'model': 'DOBLO CARGO',
 'odometer': 131648,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_e931264a-2306-5cf0-ae46-b7b71c8a6891/e931264a-2306-5cf0-ae46-b7b71c8a6891_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 327597,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/169/0f5becf14a5e41c48907f405cb68ae30/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-05-19',
 'seats_number': 2,
 'seller_name': None,
 'title': 'FIAT DOBLO CARGO (O) Cargo SX 1.3 Multijet 70kW (95CV)',
 'transmission': 'manual',
 'trim': '(O) Cargo SX 1.3 Multijet 70kW (95CV)',
 'vin': None,
 'year': 2020}
2025-12-05 20:03:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789071%27 HTTP/1.1" 200 None
2025-12-05 20:03:27 [ayvens.es] INFO: Saving data for 1789071: {'created_time': 1764965007.92053, 'last_price_update_time': 1764965007.920539, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789071') HTTP/1.1" 204 0
2025-12-05 20:03:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/409/5fc5797b95af4219a6ced3d5e6ab54f6/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/597/9d095159304349c2bb2706484298cbb8/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:27 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 1st time calling it.
2025-12-05 20:03:27 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:27 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:27 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:27 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a95b50>
2025-12-05 20:03:27 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:27 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:28 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:03:28 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:28 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:28 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:28 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_c8d8caba-95dd-5021-b550-d4c90dc918fe/c8d8caba-95dd-5021-b550-d4c90dc918fe_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F636%252F640b2208cc3442c5b727c20104939888%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/636/640b2208cc3442c5b727c20104939888/images/large/1.jpg saved to cache
2025-12-05 20:03:28 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item c8d8caba-95dd-5021-b550-d4c90dc918fe
2025-12-05 20:03:28 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: c8d8caba-95dd-5021-b550-d4c90dc918fe, skipping ID generation
2025-12-05 20:03:28 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item c8d8caba-95dd-5021-b550-d4c90dc918fe with scrape type 1
2025-12-05 20:03:28 [crawlers.middlewares.photo_download_middleware] INFO: Removing item c8d8caba-95dd-5021-b550-d4c90dc918fe from tracking dictionaries
2025-12-05 20:03:28 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item c8d8caba-95dd-5021-b550-d4c90dc918fe sending to next pipeline
2025-12-05 20:03:28 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: c8d8caba-95dd-5021-b550-d4c90dc918fe, identifier: 1789073
2025-12-05 20:03:28 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:28 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:28 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:28 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0d5f0d0>
2025-12-05 20:03:28 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:28 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:28 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1971
2025-12-05 20:03:28 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:28 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:28 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:28 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_8c2945fb-92ba-50e9-803c-41788dfb81cf/8c2945fb-92ba-50e9-803c-41788dfb81cf_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F086%252F07b2b1b1487e4beb90577a1ecf46e86a%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/086/07b2b1b1487e4beb90577a1ecf46e86a/images/large/1.jpg saved to cache
2025-12-05 20:03:28 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 8c2945fb-92ba-50e9-803c-41788dfb81cf
2025-12-05 20:03:28 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 8c2945fb-92ba-50e9-803c-41788dfb81cf, skipping ID generation
2025-12-05 20:03:28 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 8c2945fb-92ba-50e9-803c-41788dfb81cf with scrape type 1
2025-12-05 20:03:28 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 8c2945fb-92ba-50e9-803c-41788dfb81cf from tracking dictionaries
2025-12-05 20:03:28 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 8c2945fb-92ba-50e9-803c-41788dfb81cf sending to next pipeline
2025-12-05 20:03:28 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 8c2945fb-92ba-50e9-803c-41788dfb81cf, identifier: 1789072
2025-12-05 20:03:28 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: c8d8caba-95dd-5021-b550-d4c90dc918fe with identifier: 1789073
2025-12-05 20:03:28 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: c8d8caba-95dd-5021-b550-d4c90dc918fe with identifier: 1789073
2025-12-05 20:03:28 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'renault', 'model': 'megane', 'trim': 'business blue dci 81 kw (115cv)', 'title': 'renault megane business blue dci 81 kw (115cv)', 'price': 0, 'odometer': 200669, 'odometer_reading_unit': 'km', 'registration_date': '2020-10-05', 'year': 2020, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.5l', 'c_o_2_emission_value': 101, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'c8d8caba-95dd-5021-b550-d4c90dc918fe', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_c8d8caba-95dd-5021-b550-d4c90dc918fe/c8d8caba-95dd-5021-b550-d4c90dc918fe_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/636/640b2208cc3442c5b727c20104939888/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 358025}], 'current_year_diff': 5, 'age_in_months': 62, 'make_model': 'renault megane', 'sanitized_make': 'renault', 'odometer_in_km': 200669}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 62,
     'auction_title': '',
     'c_o_2_emission_value': 101,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.5l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': 'c8d8caba-95dd-5021-b550-d4c90dc918fe',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
     'listing': 'auction',
     'location': None,
     'make': 'renault',
     'make_model': 'renault megane',
     'model': 'megane',
     'odometer': 200669,
     'odometer_in_km': 200669,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_c8d8caba-95dd-5021-b550-d4c90dc918fe/c8d8caba-95dd-5021-b550-d4c90dc918fe_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 358025,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/636/640b2208cc3442c5b727c20104939888/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-10-05',
     'sanitized_make': 'renault',
     'seats_number': 5,
     'seller_name': None,
     'title': 'renault megane business blue dci 81 kw (115cv)',
     'transmission': 'manual',
     'trim': 'business blue dci 81 kw (115cv)',
     'vin': None,
     'year': 2020}
2025-12-05 20:03:28 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE failed for item: c8d8caba-95dd-5021-b550-d4c90dc918fe spider: ayvens.es
2025-12-05 20:03:29 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: c8d8caba-95dd-5021-b550-d4c90dc918fe due to rule: high_mileage.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'renault', 'model': 'megane', 'trim': 'business blue dci 81 kw (115cv)', 'title': 'renault megane business blue dci 81 kw (115cv)', 'price': 0, 'odometer': 200669, 'odometer_reading_unit': 'km', 'registration_date': '2020-10-05', 'year': 2020, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.5l', 'c_o_2_emission_value': 101, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': 'c8d8caba-95dd-5021-b550-d4c90dc918fe', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_c8d8caba-95dd-5021-b550-d4c90dc918fe/c8d8caba-95dd-5021-b550-d4c90dc918fe_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/636/640b2208cc3442c5b727c20104939888/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 358025}], 'current_year_diff': 5, 'age_in_months': 62, 'make_model': 'renault megane', 'sanitized_make': 'renault', 'odometer_in_km': 200669}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 62,
     'auction_title': '',
     'c_o_2_emission_value': 101,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 5,
     'doors_number': 5,
     'engine': '1.5l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': 'c8d8caba-95dd-5021-b550-d4c90dc918fe',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
     'listing': 'auction',
     'location': None,
     'make': 'renault',
     'make_model': 'renault megane',
     'model': 'megane',
     'odometer': 200669,
     'odometer_in_km': 200669,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_c8d8caba-95dd-5021-b550-d4c90dc918fe/c8d8caba-95dd-5021-b550-d4c90dc918fe_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 358025,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/636/640b2208cc3442c5b727c20104939888/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2020-10-05',
     'sanitized_make': 'renault',
     'seats_number': 5,
     'seller_name': None,
     'title': 'renault megane business blue dci 81 kw (115cv)',
     'transmission': 'manual',
     'trim': 'business blue dci 81 kw (115cv)',
     'vin': None,
     'year': 2020}
{'auction_title': '',
 'c_o_2_emission_value': 101,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'c8d8caba-95dd-5021-b550-d4c90dc918fe',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'MEGANE',
 'odometer': 200669,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_c8d8caba-95dd-5021-b550-d4c90dc918fe/c8d8caba-95dd-5021-b550-d4c90dc918fe_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 358025,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/636/640b2208cc3442c5b727c20104939888/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-10-05',
 'seats_number': 5,
 'seller_name': None,
 'title': 'RENAULT MEGANE Business Blue dCi 81 kW (115CV)',
 'transmission': 'manual',
 'trim': 'Business Blue dCi 81 kW (115CV)',
 'vin': None,
 'year': 2020}
2025-12-05 20:03:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789073%27 HTTP/1.1" 200 None
2025-12-05 20:03:29 [ayvens.es] INFO: Saving data for 1789073: {'created_time': 1764965009.067501, 'last_price_update_time': 1764965009.067511, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789073') HTTP/1.1" 204 0
2025-12-05 20:03:29 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 8c2945fb-92ba-50e9-803c-41788dfb81cf with identifier: 1789072
2025-12-05 20:03:29 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:03:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:03:29 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 8c2945fb-92ba-50e9-803c-41788dfb81cf with identifier: 1789072
2025-12-05 20:03:29 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'renault' containing 'renault' and model 'kangoo express' containing 'express'
2025-12-05 20:03:29 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 8c2945fb-92ba-50e9-803c-41788dfb81cf spider: ayvens.es
2025-12-05 20:03:29 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 8c2945fb-92ba-50e9-803c-41788dfb81cf due to rule: makes_models_not_allowed.json, error: Rejected due to make 'renault' containing 'renault' and model 'kangoo express' containing 'express'
{'auction_title': '',
 'c_o_2_emission_value': 112,
 'color': 'Mineral blanco opaca',
 'currency': 'EUR',
 'doors_number': 3,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '8c2945fb-92ba-50e9-803c-41788dfb81cf',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'KANGOO EXPRESS',
 'odometer': 133738,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_8c2945fb-92ba-50e9-803c-41788dfb81cf/8c2945fb-92ba-50e9-803c-41788dfb81cf_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 343615,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/086/07b2b1b1487e4beb90577a1ecf46e86a/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-01-31',
 'seats_number': 2,
 'seller_name': None,
 'title': 'RENAULT KANGOO EXPRESS (O) Profesional dCi 55 kW (75 CV)',
 'transmission': 'manual',
 'trim': '(O) Profesional dCi 55 kW (75 CV)',
 'vin': None,
 'year': 2020}
2025-12-05 20:03:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789072%27 HTTP/1.1" 200 None
2025-12-05 20:03:29 [ayvens.es] INFO: Saving data for 1789072: {'created_time': 1764965009.364321, 'last_price_update_time': 1764965009.364333, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789072') HTTP/1.1" 204 0
2025-12-05 20:03:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/983/a4f5444e64d34dd7b7663132d8f65a13/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:29 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:29 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:29 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:29 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b05fb190>
2025-12-05 20:03:29 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:29 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:29 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1969
2025-12-05 20:03:29 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:29 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:29 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:29 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_cad47282-42ae-5da8-9918-9a44e04455b1/cad47282-42ae-5da8-9918-9a44e04455b1_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F409%252F5fc5797b95af4219a6ced3d5e6ab54f6%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/409/5fc5797b95af4219a6ced3d5e6ab54f6/images/large/1.jpg saved to cache
2025-12-05 20:03:29 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item cad47282-42ae-5da8-9918-9a44e04455b1
2025-12-05 20:03:29 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: cad47282-42ae-5da8-9918-9a44e04455b1, skipping ID generation
2025-12-05 20:03:29 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item cad47282-42ae-5da8-9918-9a44e04455b1 with scrape type 1
2025-12-05 20:03:29 [crawlers.middlewares.photo_download_middleware] INFO: Removing item cad47282-42ae-5da8-9918-9a44e04455b1 from tracking dictionaries
2025-12-05 20:03:29 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item cad47282-42ae-5da8-9918-9a44e04455b1 sending to next pipeline
2025-12-05 20:03:29 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: cad47282-42ae-5da8-9918-9a44e04455b1, identifier: 1789075
2025-12-05 20:03:29 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:29 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:29 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:29 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0435850>
2025-12-05 20:03:29 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:29 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:30 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:03:30 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:30 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:30 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:30 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_f400ff90-c505-594b-b099-784624b1e330/f400ff90-c505-594b-b099-784624b1e330_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F597%252F9d095159304349c2bb2706484298cbb8%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/597/9d095159304349c2bb2706484298cbb8/images/large/1.jpg saved to cache
2025-12-05 20:03:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item f400ff90-c505-594b-b099-784624b1e330
2025-12-05 20:03:30 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: f400ff90-c505-594b-b099-784624b1e330, skipping ID generation
2025-12-05 20:03:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f400ff90-c505-594b-b099-784624b1e330 with scrape type 1
2025-12-05 20:03:30 [crawlers.middlewares.photo_download_middleware] INFO: Removing item f400ff90-c505-594b-b099-784624b1e330 from tracking dictionaries
2025-12-05 20:03:30 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item f400ff90-c505-594b-b099-784624b1e330 sending to next pipeline
2025-12-05 20:03:30 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: f400ff90-c505-594b-b099-784624b1e330, identifier: 1789074
2025-12-05 20:03:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/324/598c2a198ab5472f97322eef01f70751/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/194/6ec4aef21025451386604dc3673b1e0f/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:30 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: cad47282-42ae-5da8-9918-9a44e04455b1 with identifier: 1789075
2025-12-05 20:03:30 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: cad47282-42ae-5da8-9918-9a44e04455b1 with identifier: 1789075
2025-12-05 20:03:30 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: 133135 is greater than the maximum of 130000

Failed validating 'maximum' in schema['allOf'][0]['then']['properties']['odometer_in_km']:
    {'maximum': 130000, 'type': 'integer'}

On instance['odometer_in_km']:
    133135
2025-12-05 20:03:30 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE_FOR_MAKE failed for item: cad47282-42ae-5da8-9918-9a44e04455b1 spider: ayvens.es
2025-12-05 20:03:30 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: cad47282-42ae-5da8-9918-9a44e04455b1 due to rule: high_mileage_for_make.json, error: 133135 is greater than the maximum of 130000

Failed validating 'maximum' in schema['allOf'][0]['then']['properties']['odometer_in_km']:
    {'maximum': 130000, 'type': 'integer'}

On instance['odometer_in_km']:
    133135
{'auction_title': '',
 'c_o_2_emission_value': 93,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'cad47282-42ae-5da8-9918-9a44e04455b1',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': '308',
 'odometer': 133135,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_cad47282-42ae-5da8-9918-9a44e04455b1/cad47282-42ae-5da8-9918-9a44e04455b1_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 359146,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/409/5fc5797b95af4219a6ced3d5e6ab54f6/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2018-10-01',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT 308 5p Business Line 1.5 BlueHDi 96kW (130CV',
 'transmission': 'manual',
 'trim': '5p Business Line 1.5 BlueHDi 96kW (130CV',
 'vin': None,
 'year': 2018}
2025-12-05 20:03:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789075%27 HTTP/1.1" 200 None
2025-12-05 20:03:30 [ayvens.es] INFO: Saving data for 1789075: {'created_time': 1764965010.580051, 'last_price_update_time': 1764965010.580063, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789075') HTTP/1.1" 204 0
2025-12-05 20:03:30 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: f400ff90-c505-594b-b099-784624b1e330 with identifier: 1789074
2025-12-05 20:03:30 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: f400ff90-c505-594b-b099-784624b1e330 with identifier: 1789074
2025-12-05 20:03:30 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: f400ff90-c505-594b-b099-784624b1e330 with identifier: 1789074 to the API
2025-12-05 20:03:30 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:30 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:30 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/597/9d095159304349c2bb2706484298cbb8/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 106,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.6L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'f400ff90-c505-594b-b099-784624b1e330',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'GOLF',
 'odometer': 98791,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_f400ff90-c505-594b-b099-784624b1e330/f400ff90-c505-594b-b099-784624b1e330_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 341256,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/597/9d095159304349c2bb2706484298cbb8/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2017-10-05',
 'seats_number': 5,
 'seller_name': None,
 'title': 'VOLKSWAGEN GOLF Edition 1.6 TDI 85kW (115CV)',
 'transmission': 'manual',
 'trim': 'Edition 1.6 TDI 85kW (115CV)',
 'vin': None,
 'year': 2017}
2025-12-05 20:03:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789074%27 HTTP/1.1" 200 None
2025-12-05 20:03:31 [ayvens.es] INFO: Saving data for 1789074: {'created_time': 1764965011.077143, 'last_price_update_time': 1764965011.077153, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789074') HTTP/1.1" 204 0
2025-12-05 20:03:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/018/aa89fb84055a466c8b1feb6fad039b35/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:31 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:31 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:31 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:31 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a837d0>
2025-12-05 20:03:31 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:31 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:31 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:03:31 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:31 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:31 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:31 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_96c9774f-52bd-5ee7-b544-64b990a7a20b/96c9774f-52bd-5ee7-b544-64b990a7a20b_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F983%252Fa4f5444e64d34dd7b7663132d8f65a13%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/983/a4f5444e64d34dd7b7663132d8f65a13/images/large/1.jpg saved to cache
2025-12-05 20:03:31 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 96c9774f-52bd-5ee7-b544-64b990a7a20b
2025-12-05 20:03:31 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 96c9774f-52bd-5ee7-b544-64b990a7a20b, skipping ID generation
2025-12-05 20:03:31 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 96c9774f-52bd-5ee7-b544-64b990a7a20b with scrape type 1
2025-12-05 20:03:31 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 96c9774f-52bd-5ee7-b544-64b990a7a20b from tracking dictionaries
2025-12-05 20:03:31 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 96c9774f-52bd-5ee7-b544-64b990a7a20b sending to next pipeline
2025-12-05 20:03:31 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 96c9774f-52bd-5ee7-b544-64b990a7a20b, identifier: 1789077
2025-12-05 20:03:31 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 96c9774f-52bd-5ee7-b544-64b990a7a20b with identifier: 1789077
2025-12-05 20:03:31 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 96c9774f-52bd-5ee7-b544-64b990a7a20b with identifier: 1789077
2025-12-05 20:03:31 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'citroën' containing 'citroën' and model 'berlingo' containing 'berlingo'
2025-12-05 20:03:31 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 96c9774f-52bd-5ee7-b544-64b990a7a20b spider: ayvens.es
2025-12-05 20:03:31 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 96c9774f-52bd-5ee7-b544-64b990a7a20b due to rule: makes_models_not_allowed.json, error: Rejected due to make 'citroën' containing 'citroën' and model 'berlingo' containing 'berlingo'
{'auction_title': '',
 'c_o_2_emission_value': 107,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '96c9774f-52bd-5ee7-b544-64b990a7a20b',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'CITROËN',
 'model': 'BERLINGO',
 'odometer': 187424,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_96c9774f-52bd-5ee7-b544-64b990a7a20b/96c9774f-52bd-5ee7-b544-64b990a7a20b_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 341936,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/983/a4f5444e64d34dd7b7663132d8f65a13/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-11-16',
 'seats_number': 5,
 'seller_name': None,
 'title': 'CITROËN BERLINGO Talla M BlueHDi 100 S&S FEEL',
 'transmission': 'manual',
 'trim': 'Talla M BlueHDi 100 S&S FEEL',
 'vin': None,
 'year': 2020}
2025-12-05 20:03:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789077%27 HTTP/1.1" 200 None
2025-12-05 20:03:31 [ayvens.es] INFO: Saving data for 1789077: {'created_time': 1764965011.789401, 'last_price_update_time': 1764965011.789412, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789077') HTTP/1.1" 204 0
2025-12-05 20:03:31 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:31 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:31 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:31 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a51c10>
2025-12-05 20:03:31 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:31 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:32 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:03:32 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:32 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:32 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:32 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_882ff929-ae62-5f3d-96e9-cdbb5434aff8/882ff929-ae62-5f3d-96e9-cdbb5434aff8_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F324%252F598c2a198ab5472f97322eef01f70751%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/324/598c2a198ab5472f97322eef01f70751/images/large/1.jpg saved to cache
2025-12-05 20:03:32 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 882ff929-ae62-5f3d-96e9-cdbb5434aff8
2025-12-05 20:03:32 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 882ff929-ae62-5f3d-96e9-cdbb5434aff8, skipping ID generation
2025-12-05 20:03:32 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 882ff929-ae62-5f3d-96e9-cdbb5434aff8 with scrape type 1
2025-12-05 20:03:32 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 882ff929-ae62-5f3d-96e9-cdbb5434aff8 from tracking dictionaries
2025-12-05 20:03:32 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 882ff929-ae62-5f3d-96e9-cdbb5434aff8 sending to next pipeline
2025-12-05 20:03:32 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 882ff929-ae62-5f3d-96e9-cdbb5434aff8, identifier: 1789076
2025-12-05 20:03:32 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:32 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:32 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:32 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0521010>
2025-12-05 20:03:32 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:32 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:32 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:03:32 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:32 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:32 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:32 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_30d120c0-92b9-55a4-8742-7deb38ed42d6/30d120c0-92b9-55a4-8742-7deb38ed42d6_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F194%252F6ec4aef21025451386604dc3673b1e0f%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/194/6ec4aef21025451386604dc3673b1e0f/images/large/1.jpg saved to cache
2025-12-05 20:03:32 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 30d120c0-92b9-55a4-8742-7deb38ed42d6
2025-12-05 20:03:32 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 30d120c0-92b9-55a4-8742-7deb38ed42d6, skipping ID generation
2025-12-05 20:03:32 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 30d120c0-92b9-55a4-8742-7deb38ed42d6 with scrape type 1
2025-12-05 20:03:32 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 30d120c0-92b9-55a4-8742-7deb38ed42d6 from tracking dictionaries
2025-12-05 20:03:32 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 30d120c0-92b9-55a4-8742-7deb38ed42d6 sending to next pipeline
2025-12-05 20:03:32 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 30d120c0-92b9-55a4-8742-7deb38ed42d6, identifier: 1789078
2025-12-05 20:03:32 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 882ff929-ae62-5f3d-96e9-cdbb5434aff8 with identifier: 1789076
2025-12-05 20:03:32 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 882ff929-ae62-5f3d-96e9-cdbb5434aff8 with identifier: 1789076
2025-12-05 20:03:32 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 882ff929-ae62-5f3d-96e9-cdbb5434aff8 with identifier: 1789076 to the API
2025-12-05 20:03:32 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:32 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:33 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/324/598c2a198ab5472f97322eef01f70751/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 96,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '882ff929-ae62-5f3d-96e9-cdbb5434aff8',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'CITROËN',
 'model': 'C3',
 'odometer': 51290,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_882ff929-ae62-5f3d-96e9-cdbb5434aff8/882ff929-ae62-5f3d-96e9-cdbb5434aff8_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 466719,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/324/598c2a198ab5472f97322eef01f70751/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-04-01',
 'seats_number': 5,
 'seller_name': None,
 'title': 'CITROËN C3 BlueHDi 73KW (100CV) S&S FEEL',
 'transmission': 'manual',
 'trim': 'BlueHDi 73KW (100CV) S&S FEEL',
 'vin': None,
 'year': 2019}
2025-12-05 20:03:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789076%27 HTTP/1.1" 200 None
2025-12-05 20:03:33 [ayvens.es] INFO: Saving data for 1789076: {'created_time': 1764965013.097384, 'last_price_update_time': 1764965013.097393, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789076') HTTP/1.1" 204 0
2025-12-05 20:03:33 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 30d120c0-92b9-55a4-8742-7deb38ed42d6 with identifier: 1789078
2025-12-05 20:03:33 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 30d120c0-92b9-55a4-8742-7deb38ed42d6 with identifier: 1789078
2025-12-05 20:03:33 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'renault' containing 'renault' and model 'kangoo express' containing 'express'
2025-12-05 20:03:33 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 30d120c0-92b9-55a4-8742-7deb38ed42d6 spider: ayvens.es
2025-12-05 20:03:33 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 30d120c0-92b9-55a4-8742-7deb38ed42d6 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'renault' containing 'renault' and model 'kangoo express' containing 'express'
{'auction_title': '',
 'c_o_2_emission_value': 112,
 'color': 'Mineral blanco opaca',
 'currency': 'EUR',
 'doors_number': 3,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '30d120c0-92b9-55a4-8742-7deb38ed42d6',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'KANGOO EXPRESS',
 'odometer': 115392,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_30d120c0-92b9-55a4-8742-7deb38ed42d6/30d120c0-92b9-55a4-8742-7deb38ed42d6_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 326028,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/194/6ec4aef21025451386604dc3673b1e0f/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-02-05',
 'seats_number': 2,
 'seller_name': None,
 'title': 'RENAULT KANGOO EXPRESS (O) Profesional dCi 55 kW (75 CV)',
 'transmission': 'manual',
 'trim': '(O) Profesional dCi 55 kW (75 CV)',
 'vin': None,
 'year': 2020}
2025-12-05 20:03:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789078%27 HTTP/1.1" 200 None
2025-12-05 20:03:33 [ayvens.es] INFO: Saving data for 1789078: {'created_time': 1764965013.340616, 'last_price_update_time': 1764965013.340625, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789078') HTTP/1.1" 204 0
2025-12-05 20:03:33 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:33 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:33 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:33 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b054fe10>
2025-12-05 20:03:33 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:33 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:33 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:03:33 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:33 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:33 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:33 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_06d5c6d4-4b49-505c-bc95-20bd28d30341/06d5c6d4-4b49-505c-bc95-20bd28d30341_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F018%252Faa89fb84055a466c8b1feb6fad039b35%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/018/aa89fb84055a466c8b1feb6fad039b35/images/large/1.jpg saved to cache
2025-12-05 20:03:33 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 06d5c6d4-4b49-505c-bc95-20bd28d30341
2025-12-05 20:03:33 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 06d5c6d4-4b49-505c-bc95-20bd28d30341, skipping ID generation
2025-12-05 20:03:33 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 06d5c6d4-4b49-505c-bc95-20bd28d30341 with scrape type 1
2025-12-05 20:03:33 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 06d5c6d4-4b49-505c-bc95-20bd28d30341 from tracking dictionaries
2025-12-05 20:03:33 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 06d5c6d4-4b49-505c-bc95-20bd28d30341 sending to next pipeline
2025-12-05 20:03:33 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 06d5c6d4-4b49-505c-bc95-20bd28d30341, identifier: 1789079
2025-12-05 20:03:33 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 06d5c6d4-4b49-505c-bc95-20bd28d30341 with identifier: 1789079
2025-12-05 20:03:33 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:03:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:03:33 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 06d5c6d4-4b49-505c-bc95-20bd28d30341 with identifier: 1789079
2025-12-05 20:03:33 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: 153892 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    153892
2025-12-05 20:03:33 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE_FOR_COUNTRY failed for item: 06d5c6d4-4b49-505c-bc95-20bd28d30341 spider: ayvens.es
2025-12-05 20:03:34 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 06d5c6d4-4b49-505c-bc95-20bd28d30341 due to rule: high_mileage_for_country.json, error: 153892 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    153892
{'auction_title': '',
 'c_o_2_emission_value': 147,
 'color': 'Madrid',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '06d5c6d4-4b49-505c-bc95-20bd28d30341',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'SKODA',
 'model': 'KAROQ',
 'odometer': 153892,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_06d5c6d4-4b49-505c-bc95-20bd28d30341/06d5c6d4-4b49-505c-bc95-20bd28d30341_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 353936,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/018/aa89fb84055a466c8b1feb6fad039b35/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2022-03-21',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SKODA KAROQ 2.0 TDI 110kW (150CV) DSG 4X4 Ambition',
 'transmission': 'manual',
 'trim': '2.0 TDI 110kW (150CV) DSG 4X4 Ambition',
 'vin': None,
 'year': 2022}
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789079%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [ayvens.es] INFO: Saving data for 1789079: {'created_time': 1764965014.076098, 'last_price_update_time': 1764965014.076106, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789079') HTTP/1.1" 204 0
2025-12-05 20:03:34 [scrapy.core.engine] DEBUG: Crawled (200) <POST https://api.aldcarmarket.com/graphql/saleevents> (referer: https://carmarket.ayvens.com/) ['zyte-api']
2025-12-05 20:03:34 [ayvens.es] INFO: Found listing with ID: 1789080
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789080%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [ayvens.es] INFO: Scrape type for 1789080: 1
2025-12-05 20:03:34 [ayvens.es] INFO: Extracting data for car ID: 1789080
2025-12-05 20:03:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: a0f6c4d7-9ed7-55d6-8b8e-f417b322c353 with identifier: 1789080
2025-12-05 20:03:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a0f6c4d7-9ed7-55d6-8b8e-f417b322c353 with scrape type 1
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item a0f6c4d7-9ed7-55d6-8b8e-f417b322c353
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F074%252Ff496c58638ad49cca049fdf2c07cb71b%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/074/f496c58638ad49cca049fdf2c07cb71b/images/large/1.jpg not downloaded yet
2025-12-05 20:03:34 [ayvens.es] INFO: Found listing with ID: 1789081
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789081%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [ayvens.es] INFO: Scrape type for 1789081: 1
2025-12-05 20:03:34 [ayvens.es] INFO: Extracting data for car ID: 1789081
2025-12-05 20:03:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: b1924d46-fa9e-5d85-9bca-a0f842250075 with identifier: 1789081
2025-12-05 20:03:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b1924d46-fa9e-5d85-9bca-a0f842250075 with scrape type 1
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item b1924d46-fa9e-5d85-9bca-a0f842250075
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F006%252Ff1e8e61f197243e1b9c4206396e66d07%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/006/f1e8e61f197243e1b9c4206396e66d07/images/large/1.jpg not downloaded yet
2025-12-05 20:03:34 [ayvens.es] INFO: Found listing with ID: 1789082
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789082%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [ayvens.es] INFO: Scrape type for 1789082: 1
2025-12-05 20:03:34 [ayvens.es] INFO: Extracting data for car ID: 1789082
2025-12-05 20:03:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 9bef9935-26f6-53ae-be85-4f9568e3dee7 with identifier: 1789082
2025-12-05 20:03:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9bef9935-26f6-53ae-be85-4f9568e3dee7 with scrape type 1
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 9bef9935-26f6-53ae-be85-4f9568e3dee7
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F404%252F7ce731af5a28433780d56df5789fc50a%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/404/7ce731af5a28433780d56df5789fc50a/images/large/1.jpg not downloaded yet
2025-12-05 20:03:34 [ayvens.es] INFO: Found listing with ID: 1789083
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789083%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [ayvens.es] INFO: Scrape type for 1789083: 1
2025-12-05 20:03:34 [ayvens.es] INFO: Extracting data for car ID: 1789083
2025-12-05 20:03:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 2b1f18ae-2969-55c4-acb9-7c0b7b64e4da with identifier: 1789083
2025-12-05 20:03:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 2b1f18ae-2969-55c4-acb9-7c0b7b64e4da with scrape type 1
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 2b1f18ae-2969-55c4-acb9-7c0b7b64e4da
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F500%252Fd6659fec8bfb451d989b391e4b215f6a%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/500/d6659fec8bfb451d989b391e4b215f6a/images/large/1.jpg not downloaded yet
2025-12-05 20:03:34 [ayvens.es] INFO: Found listing with ID: 1789084
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789084%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [ayvens.es] INFO: Scrape type for 1789084: 1
2025-12-05 20:03:34 [ayvens.es] INFO: Extracting data for car ID: 1789084
2025-12-05 20:03:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: d4571aa6-9e0c-5ae0-9b43-e4a674ae97c4 with identifier: 1789084
2025-12-05 20:03:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item d4571aa6-9e0c-5ae0-9b43-e4a674ae97c4 with scrape type 1
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item d4571aa6-9e0c-5ae0-9b43-e4a674ae97c4
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F352%252Ff7df2e0a389d492789a5e2e4b965515e%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/352/f7df2e0a389d492789a5e2e4b965515e/images/large/1.jpg not downloaded yet
2025-12-05 20:03:34 [ayvens.es] INFO: Found listing with ID: 1789085
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789085%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [ayvens.es] INFO: Scrape type for 1789085: 1
2025-12-05 20:03:34 [ayvens.es] INFO: Extracting data for car ID: 1789085
2025-12-05 20:03:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 0c3c8ff4-52f2-541a-a6d2-5d1beec315b7 with identifier: 1789085
2025-12-05 20:03:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 0c3c8ff4-52f2-541a-a6d2-5d1beec315b7 with scrape type 1
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 0c3c8ff4-52f2-541a-a6d2-5d1beec315b7
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F641%252F0b94160b0f8e4d6ea96cfe0b570391ca%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/641/0b94160b0f8e4d6ea96cfe0b570391ca/images/large/1.jpg not downloaded yet
2025-12-05 20:03:34 [ayvens.es] INFO: Found listing with ID: 1789086
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789086%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [ayvens.es] INFO: Scrape type for 1789086: 1
2025-12-05 20:03:34 [ayvens.es] INFO: Extracting data for car ID: 1789086
2025-12-05 20:03:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: b6036204-7a8f-511b-8e0e-20a08deb58b8 with identifier: 1789086
2025-12-05 20:03:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b6036204-7a8f-511b-8e0e-20a08deb58b8 with scrape type 1
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item b6036204-7a8f-511b-8e0e-20a08deb58b8
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F760%252Fe595fb00fc094e88b2a9de66e06c300d%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/760/e595fb00fc094e88b2a9de66e06c300d/images/large/1.jpg not downloaded yet
2025-12-05 20:03:34 [ayvens.es] INFO: Found listing with ID: 1789087
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789087%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [ayvens.es] INFO: Scrape type for 1789087: 1
2025-12-05 20:03:34 [ayvens.es] INFO: Extracting data for car ID: 1789087
2025-12-05 20:03:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: cee3693d-47c1-54d4-912f-503f4dbe9a77 with identifier: 1789087
2025-12-05 20:03:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item cee3693d-47c1-54d4-912f-503f4dbe9a77 with scrape type 1
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item cee3693d-47c1-54d4-912f-503f4dbe9a77
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F534%252Fce2f994fcd7f4babba058cfad877d5d9%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/534/ce2f994fcd7f4babba058cfad877d5d9/images/large/1.jpg not downloaded yet
2025-12-05 20:03:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/006/f1e8e61f197243e1b9c4206396e66d07/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:34 [ayvens.es] INFO: Found listing with ID: 1789088
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789088%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [ayvens.es] INFO: Scrape type for 1789088: 1
2025-12-05 20:03:34 [ayvens.es] INFO: Extracting data for car ID: 1789088
2025-12-05 20:03:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: c9030dd5-8142-58e5-b54c-5f4202d63b21 with identifier: 1789088
2025-12-05 20:03:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item c9030dd5-8142-58e5-b54c-5f4202d63b21 with scrape type 1
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item c9030dd5-8142-58e5-b54c-5f4202d63b21
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F594%252F322c958ac29d4d7f86d13bcb174ffe76%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/594/322c958ac29d4d7f86d13bcb174ffe76/images/large/1.jpg not downloaded yet
2025-12-05 20:03:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/074/f496c58638ad49cca049fdf2c07cb71b/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:34 [ayvens.es] INFO: Found listing with ID: 1789089
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789089%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [ayvens.es] INFO: Scrape type for 1789089: 1
2025-12-05 20:03:34 [ayvens.es] INFO: Extracting data for car ID: 1789089
2025-12-05 20:03:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: a41a4c9b-2f22-5564-878f-c6d18bb49bce with identifier: 1789089
2025-12-05 20:03:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a41a4c9b-2f22-5564-878f-c6d18bb49bce with scrape type 1
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item a41a4c9b-2f22-5564-878f-c6d18bb49bce
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F656%252F525b650b54cf424b8d77095a86450987%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/656/525b650b54cf424b8d77095a86450987/images/large/1.jpg not downloaded yet
2025-12-05 20:03:34 [ayvens.es] INFO: Found listing with ID: 1789090
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789090%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [ayvens.es] INFO: Scrape type for 1789090: 1
2025-12-05 20:03:34 [ayvens.es] INFO: Extracting data for car ID: 1789090
2025-12-05 20:03:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 55c967be-cc84-53df-92dc-6f3267fef603 with identifier: 1789090
2025-12-05 20:03:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 55c967be-cc84-53df-92dc-6f3267fef603 with scrape type 1
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 55c967be-cc84-53df-92dc-6f3267fef603
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F179%252F3a615519dd044728a7019fef65772a61%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/179/3a615519dd044728a7019fef65772a61/images/large/1.jpg not downloaded yet
2025-12-05 20:03:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/404/7ce731af5a28433780d56df5789fc50a/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/500/d6659fec8bfb451d989b391e4b215f6a/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:34 [ayvens.es] INFO: Found listing with ID: 1789091
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789091%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [ayvens.es] INFO: Scrape type for 1789091: 1
2025-12-05 20:03:34 [ayvens.es] INFO: Extracting data for car ID: 1789091
2025-12-05 20:03:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: f1f8c897-6845-5f95-8b7a-229e8a6c2798 with identifier: 1789091
2025-12-05 20:03:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f1f8c897-6845-5f95-8b7a-229e8a6c2798 with scrape type 1
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item f1f8c897-6845-5f95-8b7a-229e8a6c2798
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F554%252Fa1ddbd2fb11545b29f8f09d3ecd02c93%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/554/a1ddbd2fb11545b29f8f09d3ecd02c93/images/large/1.jpg not downloaded yet
2025-12-05 20:03:34 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:34 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:34 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:34 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b046a950>
2025-12-05 20:03:34 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:34 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:34 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:35 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_b1924d46-fa9e-5d85-9bca-a0f842250075/b1924d46-fa9e-5d85-9bca-a0f842250075_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F006%252Ff1e8e61f197243e1b9c4206396e66d07%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/006/f1e8e61f197243e1b9c4206396e66d07/images/large/1.jpg saved to cache
2025-12-05 20:03:35 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item b1924d46-fa9e-5d85-9bca-a0f842250075
2025-12-05 20:03:35 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: b1924d46-fa9e-5d85-9bca-a0f842250075, skipping ID generation
2025-12-05 20:03:35 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b1924d46-fa9e-5d85-9bca-a0f842250075 with scrape type 1
2025-12-05 20:03:35 [crawlers.middlewares.photo_download_middleware] INFO: Removing item b1924d46-fa9e-5d85-9bca-a0f842250075 from tracking dictionaries
2025-12-05 20:03:35 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item b1924d46-fa9e-5d85-9bca-a0f842250075 sending to next pipeline
2025-12-05 20:03:35 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: b1924d46-fa9e-5d85-9bca-a0f842250075, identifier: 1789081
2025-12-05 20:03:35 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/352/f7df2e0a389d492789a5e2e4b965515e/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:35 [ayvens.es] INFO: Found listing with ID: 1789092
2025-12-05 20:03:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789092%27 HTTP/1.1" 200 None
2025-12-05 20:03:35 [ayvens.es] INFO: Scrape type for 1789092: 1
2025-12-05 20:03:35 [ayvens.es] INFO: Extracting data for car ID: 1789092
2025-12-05 20:03:35 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: b70ecdd8-948b-5dce-8278-e86ef5ccf632 with identifier: 1789092
2025-12-05 20:03:35 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b70ecdd8-948b-5dce-8278-e86ef5ccf632 with scrape type 1
2025-12-05 20:03:35 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item b70ecdd8-948b-5dce-8278-e86ef5ccf632
2025-12-05 20:03:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F507%252F17428490c4bb446eb2e65667dbf95cce%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/507/17428490c4bb446eb2e65667dbf95cce/images/large/1.jpg not downloaded yet
2025-12-05 20:03:35 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: b1924d46-fa9e-5d85-9bca-a0f842250075 with identifier: 1789081
2025-12-05 20:03:35 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: b1924d46-fa9e-5d85-9bca-a0f842250075 with identifier: 1789081
2025-12-05 20:03:35 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: b1924d46-fa9e-5d85-9bca-a0f842250075 with identifier: 1789081 to the API
2025-12-05 20:03:35 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:35 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:35 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/006/f1e8e61f197243e1b9c4206396e66d07/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 129,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': 'b1924d46-fa9e-5d85-9bca-a0f842250075',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'SEAT',
 'model': 'ATECA',
 'odometer': 62887,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_b1924d46-fa9e-5d85-9bca-a0f842250075/b1924d46-fa9e-5d85-9bca-a0f842250075_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 352951,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/006/f1e8e61f197243e1b9c4206396e66d07/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-03-28',
 'seats_number': 5,
 'seller_name': None,
 'title': 'SEAT ATECA 1.5 TSI 110kW DSG S&S Style Edition Nav',
 'transmission': 'manual',
 'trim': '1.5 TSI 110kW DSG S&S Style Edition Nav',
 'vin': None,
 'year': 2019}
2025-12-05 20:03:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789081%27 HTTP/1.1" 200 None
2025-12-05 20:03:35 [ayvens.es] INFO: Saving data for 1789081: {'created_time': 1764965015.647643, 'last_price_update_time': 1764965015.647654, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789081') HTTP/1.1" 204 0
2025-12-05 20:03:35 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/641/0b94160b0f8e4d6ea96cfe0b570391ca/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:35 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:35 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:35 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:35 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b053e290>
2025-12-05 20:03:35 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:35 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:36 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:03:36 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:36 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:36 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:36 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_a0f6c4d7-9ed7-55d6-8b8e-f417b322c353/a0f6c4d7-9ed7-55d6-8b8e-f417b322c353_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F074%252Ff496c58638ad49cca049fdf2c07cb71b%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:36 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/074/f496c58638ad49cca049fdf2c07cb71b/images/large/1.jpg saved to cache
2025-12-05 20:03:36 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item a0f6c4d7-9ed7-55d6-8b8e-f417b322c353
2025-12-05 20:03:36 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: a0f6c4d7-9ed7-55d6-8b8e-f417b322c353, skipping ID generation
2025-12-05 20:03:36 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a0f6c4d7-9ed7-55d6-8b8e-f417b322c353 with scrape type 1
2025-12-05 20:03:36 [crawlers.middlewares.photo_download_middleware] INFO: Removing item a0f6c4d7-9ed7-55d6-8b8e-f417b322c353 from tracking dictionaries
2025-12-05 20:03:36 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item a0f6c4d7-9ed7-55d6-8b8e-f417b322c353 sending to next pipeline
2025-12-05 20:03:36 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: a0f6c4d7-9ed7-55d6-8b8e-f417b322c353, identifier: 1789080
2025-12-05 20:03:36 [ayvens.es] INFO: Found listing with ID: 1789093
2025-12-05 20:03:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789093%27 HTTP/1.1" 200 None
2025-12-05 20:03:36 [ayvens.es] INFO: Scrape type for 1789093: 1
2025-12-05 20:03:36 [ayvens.es] INFO: Extracting data for car ID: 1789093
2025-12-05 20:03:36 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: a43cf8e1-fd1b-5a35-ae8a-ac8bfc6449f2 with identifier: 1789093
2025-12-05 20:03:36 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a43cf8e1-fd1b-5a35-ae8a-ac8bfc6449f2 with scrape type 1
2025-12-05 20:03:36 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item a43cf8e1-fd1b-5a35-ae8a-ac8bfc6449f2
2025-12-05 20:03:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F609%252F5c98d376f18f44efb9ebba4bbf1bd8b5%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:36 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/609/5c98d376f18f44efb9ebba4bbf1bd8b5/images/large/1.jpg not downloaded yet
2025-12-05 20:03:36 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: a0f6c4d7-9ed7-55d6-8b8e-f417b322c353 with identifier: 1789080
2025-12-05 20:03:36 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: a0f6c4d7-9ed7-55d6-8b8e-f417b322c353 with identifier: 1789080
2025-12-05 20:03:36 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: a0f6c4d7-9ed7-55d6-8b8e-f417b322c353 with identifier: 1789080 to the API
2025-12-05 20:03:36 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:36 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:36 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/074/f496c58638ad49cca049fdf2c07cb71b/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 120,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': 'a0f6c4d7-9ed7-55d6-8b8e-f417b322c353',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'POLO',
 'odometer': 39981,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_a0f6c4d7-9ed7-55d6-8b8e-f417b322c353/a0f6c4d7-9ed7-55d6-8b8e-f417b322c353_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 321966,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/074/f496c58638ad49cca049fdf2c07cb71b/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-04-29',
 'seats_number': 5,
 'seller_name': None,
 'title': 'VOLKSWAGEN POLO Advance 1.0 TSI 70kW (95CV)',
 'transmission': 'manual',
 'trim': 'Advance 1.0 TSI 70kW (95CV)',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789080%27 HTTP/1.1" 200 None
2025-12-05 20:03:36 [ayvens.es] INFO: Saving data for 1789080: {'created_time': 1764965016.791345, 'last_price_update_time': 1764965016.791354, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789080') HTTP/1.1" 204 0
2025-12-05 20:03:36 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:36 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:36 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:36 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0473590>
2025-12-05 20:03:36 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:36 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:36 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:03:36 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:37 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:37 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:37 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_9bef9935-26f6-53ae-be85-4f9568e3dee7/9bef9935-26f6-53ae-be85-4f9568e3dee7_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F404%252F7ce731af5a28433780d56df5789fc50a%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:37 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/404/7ce731af5a28433780d56df5789fc50a/images/large/1.jpg saved to cache
2025-12-05 20:03:37 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 9bef9935-26f6-53ae-be85-4f9568e3dee7
2025-12-05 20:03:37 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 9bef9935-26f6-53ae-be85-4f9568e3dee7, skipping ID generation
2025-12-05 20:03:37 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9bef9935-26f6-53ae-be85-4f9568e3dee7 with scrape type 1
2025-12-05 20:03:37 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 9bef9935-26f6-53ae-be85-4f9568e3dee7 from tracking dictionaries
2025-12-05 20:03:37 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 9bef9935-26f6-53ae-be85-4f9568e3dee7 sending to next pipeline
2025-12-05 20:03:37 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 9bef9935-26f6-53ae-be85-4f9568e3dee7, identifier: 1789082
2025-12-05 20:03:37 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:37 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:37 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:37 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0405dd0>
2025-12-05 20:03:37 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:37 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:37 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:03:37 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:37 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:37 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:37 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_2b1f18ae-2969-55c4-acb9-7c0b7b64e4da/2b1f18ae-2969-55c4-acb9-7c0b7b64e4da_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F500%252Fd6659fec8bfb451d989b391e4b215f6a%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:37 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/500/d6659fec8bfb451d989b391e4b215f6a/images/large/1.jpg saved to cache
2025-12-05 20:03:37 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 2b1f18ae-2969-55c4-acb9-7c0b7b64e4da
2025-12-05 20:03:37 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 2b1f18ae-2969-55c4-acb9-7c0b7b64e4da, skipping ID generation
2025-12-05 20:03:37 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 2b1f18ae-2969-55c4-acb9-7c0b7b64e4da with scrape type 1
2025-12-05 20:03:37 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 2b1f18ae-2969-55c4-acb9-7c0b7b64e4da from tracking dictionaries
2025-12-05 20:03:37 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 2b1f18ae-2969-55c4-acb9-7c0b7b64e4da sending to next pipeline
2025-12-05 20:03:37 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 2b1f18ae-2969-55c4-acb9-7c0b7b64e4da, identifier: 1789083
2025-12-05 20:03:37 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:37 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:37 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:37 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0568610>
2025-12-05 20:03:37 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:37 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:37 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:03:37 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:37 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:37 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:37 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_d4571aa6-9e0c-5ae0-9b43-e4a674ae97c4/d4571aa6-9e0c-5ae0-9b43-e4a674ae97c4_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F352%252Ff7df2e0a389d492789a5e2e4b965515e%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:37 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/352/f7df2e0a389d492789a5e2e4b965515e/images/large/1.jpg saved to cache
2025-12-05 20:03:37 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item d4571aa6-9e0c-5ae0-9b43-e4a674ae97c4
2025-12-05 20:03:37 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: d4571aa6-9e0c-5ae0-9b43-e4a674ae97c4, skipping ID generation
2025-12-05 20:03:37 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item d4571aa6-9e0c-5ae0-9b43-e4a674ae97c4 with scrape type 1
2025-12-05 20:03:37 [crawlers.middlewares.photo_download_middleware] INFO: Removing item d4571aa6-9e0c-5ae0-9b43-e4a674ae97c4 from tracking dictionaries
2025-12-05 20:03:37 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item d4571aa6-9e0c-5ae0-9b43-e4a674ae97c4 sending to next pipeline
2025-12-05 20:03:38 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: d4571aa6-9e0c-5ae0-9b43-e4a674ae97c4, identifier: 1789084
2025-12-05 20:03:38 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/760/e595fb00fc094e88b2a9de66e06c300d/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:38 [ayvens.es] INFO: Found listing with ID: 1789094
2025-12-05 20:03:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789094%27 HTTP/1.1" 200 None
2025-12-05 20:03:38 [ayvens.es] INFO: Scrape type for 1789094: 1
2025-12-05 20:03:38 [ayvens.es] INFO: Extracting data for car ID: 1789094
2025-12-05 20:03:38 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 44cf5a86-4384-5f1e-a539-acb64af804af with identifier: 1789094
2025-12-05 20:03:38 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 44cf5a86-4384-5f1e-a539-acb64af804af with scrape type 1
2025-12-05 20:03:38 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 44cf5a86-4384-5f1e-a539-acb64af804af
2025-12-05 20:03:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F464%252F003f485f8f704b729c9637118d40e078%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:38 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/464/003f485f8f704b729c9637118d40e078/images/large/1.jpg not downloaded yet
2025-12-05 20:03:38 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 9bef9935-26f6-53ae-be85-4f9568e3dee7 with identifier: 1789082
2025-12-05 20:03:38 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 9bef9935-26f6-53ae-be85-4f9568e3dee7 with identifier: 1789082
2025-12-05 20:03:38 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'citroën' containing 'citroën' and model 'berlingo' containing 'berlingo'
2025-12-05 20:03:38 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 9bef9935-26f6-53ae-be85-4f9568e3dee7 spider: ayvens.es
2025-12-05 20:03:38 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 9bef9935-26f6-53ae-be85-4f9568e3dee7 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'citroën' containing 'citroën' and model 'berlingo' containing 'berlingo'
{'auction_title': '',
 'c_o_2_emission_value': 135,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '9bef9935-26f6-53ae-be85-4f9568e3dee7',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'CITROËN',
 'model': 'BERLINGO',
 'odometer': 107178,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_9bef9935-26f6-53ae-be85-4f9568e3dee7/9bef9935-26f6-53ae-be85-4f9568e3dee7_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 339572,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/404/7ce731af5a28433780d56df5789fc50a/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-07-15',
 'seats_number': 5,
 'seller_name': None,
 'title': 'CITROËN BERLINGO Talla M BlueHDi 100 S&S FEEL',
 'transmission': 'manual',
 'trim': 'Talla M BlueHDi 100 S&S FEEL',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789082%27 HTTP/1.1" 200 None
2025-12-05 20:03:38 [ayvens.es] INFO: Saving data for 1789082: {'created_time': 1764965018.362952, 'last_price_update_time': 1764965018.362961, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789082') HTTP/1.1" 204 0
2025-12-05 20:03:38 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 2b1f18ae-2969-55c4-acb9-7c0b7b64e4da with identifier: 1789083
2025-12-05 20:03:38 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 2b1f18ae-2969-55c4-acb9-7c0b7b64e4da with identifier: 1789083
2025-12-05 20:03:38 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'citroën' containing 'citroën' and model 'berlingo' containing 'berlingo'
2025-12-05 20:03:38 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 2b1f18ae-2969-55c4-acb9-7c0b7b64e4da spider: ayvens.es
2025-12-05 20:03:38 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 2b1f18ae-2969-55c4-acb9-7c0b7b64e4da due to rule: makes_models_not_allowed.json, error: Rejected due to make 'citroën' containing 'citroën' and model 'berlingo' containing 'berlingo'
{'auction_title': '',
 'c_o_2_emission_value': 139,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 3,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '2b1f18ae-2969-55c4-acb9-7c0b7b64e4da',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'CITROËN',
 'model': 'BERLINGO',
 'odometer': 77009,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_2b1f18ae-2969-55c4-acb9-7c0b7b64e4da/2b1f18ae-2969-55c4-acb9-7c0b7b64e4da_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 330753,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/500/d6659fec8bfb451d989b391e4b215f6a/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-07-02',
 'seats_number': 2,
 'seller_name': None,
 'title': 'CITROËN BERLINGO Talla M BlueHDi 100 CONTROL',
 'transmission': 'manual',
 'trim': 'Talla M BlueHDi 100 CONTROL',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789083%27 HTTP/1.1" 200 None
2025-12-05 20:03:38 [ayvens.es] INFO: Saving data for 1789083: {'created_time': 1764965018.612179, 'last_price_update_time': 1764965018.612189, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789083') HTTP/1.1" 204 0
2025-12-05 20:03:38 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: d4571aa6-9e0c-5ae0-9b43-e4a674ae97c4 with identifier: 1789084
2025-12-05 20:03:38 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: d4571aa6-9e0c-5ae0-9b43-e4a674ae97c4 with identifier: 1789084
2025-12-05 20:03:38 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: 167883 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    167883
2025-12-05 20:03:38 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE_FOR_COUNTRY failed for item: d4571aa6-9e0c-5ae0-9b43-e4a674ae97c4 spider: ayvens.es
2025-12-05 20:03:38 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: d4571aa6-9e0c-5ae0-9b43-e4a674ae97c4 due to rule: high_mileage_for_country.json, error: 167883 is greater than the maximum of 150000

Failed validating 'maximum' in schema['allOf'][1]['then']['properties']['odometer_in_km']:
    {'maximum': 150000, 'type': 'integer'}

On instance['odometer_in_km']:
    167883
{'auction_title': '',
 'c_o_2_emission_value': 201,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.8L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'd4571aa6-9e0c-5ae0-9b43-e4a674ae97c4',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'TOYOTA',
 'model': 'LANDCRUISER',
 'odometer': 167883,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_d4571aa6-9e0c-5ae0-9b43-e4a674ae97c4/d4571aa6-9e0c-5ae0-9b43-e4a674ae97c4_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 349767,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/352/f7df2e0a389d492789a5e2e4b965515e/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-05-20',
 'seats_number': 5,
 'seller_name': None,
 'title': 'TOYOTA LANDCRUISER 2.8 D-4D GX',
 'transmission': 'manual',
 'trim': '2.8 D-4D GX',
 'vin': None,
 'year': 2020}
2025-12-05 20:03:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789084%27 HTTP/1.1" 200 None
2025-12-05 20:03:38 [ayvens.es] INFO: Saving data for 1789084: {'created_time': 1764965018.856405, 'last_price_update_time': 1764965018.856415, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789084') HTTP/1.1" 204 0
2025-12-05 20:03:38 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:38 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:38 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:38 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b71fd0>
2025-12-05 20:03:38 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:38 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:39 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:03:39 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:39 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:39 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:39 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_0c3c8ff4-52f2-541a-a6d2-5d1beec315b7/0c3c8ff4-52f2-541a-a6d2-5d1beec315b7_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F641%252F0b94160b0f8e4d6ea96cfe0b570391ca%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:39 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/641/0b94160b0f8e4d6ea96cfe0b570391ca/images/large/1.jpg saved to cache
2025-12-05 20:03:39 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 0c3c8ff4-52f2-541a-a6d2-5d1beec315b7
2025-12-05 20:03:39 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 0c3c8ff4-52f2-541a-a6d2-5d1beec315b7, skipping ID generation
2025-12-05 20:03:39 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 0c3c8ff4-52f2-541a-a6d2-5d1beec315b7 with scrape type 1
2025-12-05 20:03:39 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 0c3c8ff4-52f2-541a-a6d2-5d1beec315b7 from tracking dictionaries
2025-12-05 20:03:39 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 0c3c8ff4-52f2-541a-a6d2-5d1beec315b7 sending to next pipeline
2025-12-05 20:03:39 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 0c3c8ff4-52f2-541a-a6d2-5d1beec315b7, identifier: 1789085
2025-12-05 20:03:39 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/594/322c958ac29d4d7f86d13bcb174ffe76/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:39 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/534/ce2f994fcd7f4babba058cfad877d5d9/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:39 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/656/525b650b54cf424b8d77095a86450987/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:39 [ayvens.es] INFO: Found listing with ID: 1789095
2025-12-05 20:03:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789095%27 HTTP/1.1" 200 None
2025-12-05 20:03:39 [ayvens.es] INFO: Scrape type for 1789095: 1
2025-12-05 20:03:39 [ayvens.es] INFO: Extracting data for car ID: 1789095
2025-12-05 20:03:39 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: ff3549f8-543f-5a48-9497-3ca212b2bd28 with identifier: 1789095
2025-12-05 20:03:39 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ff3549f8-543f-5a48-9497-3ca212b2bd28 with scrape type 1
2025-12-05 20:03:39 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item ff3549f8-543f-5a48-9497-3ca212b2bd28
2025-12-05 20:03:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F851%252F372f90268e51412bae0483c8d4feff15%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:39 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/851/372f90268e51412bae0483c8d4feff15/images/large/1.jpg not downloaded yet
2025-12-05 20:03:39 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 0c3c8ff4-52f2-541a-a6d2-5d1beec315b7 with identifier: 1789085
2025-12-05 20:03:39 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 0c3c8ff4-52f2-541a-a6d2-5d1beec315b7 with identifier: 1789085
2025-12-05 20:03:39 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'renault', 'model': 'clio', 'trim': 'business energy dci 75', 'title': 'renault clio business energy dci 75', 'price': 0, 'odometer': 319158, 'odometer_reading_unit': 'km', 'registration_date': '2017-09-27', 'year': 2017, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.5l', 'c_o_2_emission_value': 85, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '0c3c8ff4-52f2-541a-a6d2-5d1beec315b7', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_0c3c8ff4-52f2-541a-a6d2-5d1beec315b7/0c3c8ff4-52f2-541a-a6d2-5d1beec315b7_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/641/0b94160b0f8e4d6ea96cfe0b570391ca/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 350704}], 'current_year_diff': 8, 'age_in_months': 98, 'make_model': 'renault clio', 'sanitized_make': 'renault', 'odometer_in_km': 319158}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 98,
     'auction_title': '',
     'c_o_2_emission_value': 85,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 8,
     'doors_number': 5,
     'engine': '1.5l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': '0c3c8ff4-52f2-541a-a6d2-5d1beec315b7',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
     'listing': 'auction',
     'location': None,
     'make': 'renault',
     'make_model': 'renault clio',
     'model': 'clio',
     'odometer': 319158,
     'odometer_in_km': 319158,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_0c3c8ff4-52f2-541a-a6d2-5d1beec315b7/0c3c8ff4-52f2-541a-a6d2-5d1beec315b7_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 350704,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/641/0b94160b0f8e4d6ea96cfe0b570391ca/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2017-09-27',
     'sanitized_make': 'renault',
     'seats_number': 5,
     'seller_name': None,
     'title': 'renault clio business energy dci 75',
     'transmission': 'manual',
     'trim': 'business energy dci 75',
     'vin': None,
     'year': 2017}
2025-12-05 20:03:39 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE failed for item: 0c3c8ff4-52f2-541a-a6d2-5d1beec315b7 spider: ayvens.es
2025-12-05 20:03:39 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 0c3c8ff4-52f2-541a-a6d2-5d1beec315b7 due to rule: high_mileage.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'renault', 'model': 'clio', 'trim': 'business energy dci 75', 'title': 'renault clio business energy dci 75', 'price': 0, 'odometer': 319158, 'odometer_reading_unit': 'km', 'registration_date': '2017-09-27', 'year': 2017, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 5, 'vin': None, 'license_plate': None, 'engine': '1.5l', 'c_o_2_emission_value': 85, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '0c3c8ff4-52f2-541a-a6d2-5d1beec315b7', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_0c3c8ff4-52f2-541a-a6d2-5d1beec315b7/0c3c8ff4-52f2-541a-a6d2-5d1beec315b7_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/641/0b94160b0f8e4d6ea96cfe0b570391ca/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 350704}], 'current_year_diff': 8, 'age_in_months': 98, 'make_model': 'renault clio', 'sanitized_make': 'renault', 'odometer_in_km': 319158}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 98,
     'auction_title': '',
     'c_o_2_emission_value': 85,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 8,
     'doors_number': 5,
     'engine': '1.5l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': '0c3c8ff4-52f2-541a-a6d2-5d1beec315b7',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
     'listing': 'auction',
     'location': None,
     'make': 'renault',
     'make_model': 'renault clio',
     'model': 'clio',
     'odometer': 319158,
     'odometer_in_km': 319158,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_0c3c8ff4-52f2-541a-a6d2-5d1beec315b7/0c3c8ff4-52f2-541a-a6d2-5d1beec315b7_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 350704,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/641/0b94160b0f8e4d6ea96cfe0b570391ca/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2017-09-27',
     'sanitized_make': 'renault',
     'seats_number': 5,
     'seller_name': None,
     'title': 'renault clio business energy dci 75',
     'transmission': 'manual',
     'trim': 'business energy dci 75',
     'vin': None,
     'year': 2017}
{'auction_title': '',
 'c_o_2_emission_value': 85,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '0c3c8ff4-52f2-541a-a6d2-5d1beec315b7',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'CLIO',
 'odometer': 319158,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_0c3c8ff4-52f2-541a-a6d2-5d1beec315b7/0c3c8ff4-52f2-541a-a6d2-5d1beec315b7_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 350704,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/641/0b94160b0f8e4d6ea96cfe0b570391ca/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2017-09-27',
 'seats_number': 5,
 'seller_name': None,
 'title': 'RENAULT CLIO Business Energy dCi 75',
 'transmission': 'manual',
 'trim': 'Business Energy dCi 75',
 'vin': None,
 'year': 2017}
2025-12-05 20:03:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789085%27 HTTP/1.1" 200 None
2025-12-05 20:03:39 [ayvens.es] INFO: Saving data for 1789085: {'created_time': 1764965019.799029, 'last_price_update_time': 1764965019.799039, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789085') HTTP/1.1" 204 0
2025-12-05 20:03:39 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:39 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:39 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:39 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0cf8610>
2025-12-05 20:03:39 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:39 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:40 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:03:40 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:40 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:40 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:40 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_b6036204-7a8f-511b-8e0e-20a08deb58b8/b6036204-7a8f-511b-8e0e-20a08deb58b8_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F760%252Fe595fb00fc094e88b2a9de66e06c300d%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:40 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/760/e595fb00fc094e88b2a9de66e06c300d/images/large/1.jpg saved to cache
2025-12-05 20:03:40 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item b6036204-7a8f-511b-8e0e-20a08deb58b8
2025-12-05 20:03:40 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: b6036204-7a8f-511b-8e0e-20a08deb58b8, skipping ID generation
2025-12-05 20:03:40 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b6036204-7a8f-511b-8e0e-20a08deb58b8 with scrape type 1
2025-12-05 20:03:40 [crawlers.middlewares.photo_download_middleware] INFO: Removing item b6036204-7a8f-511b-8e0e-20a08deb58b8 from tracking dictionaries
2025-12-05 20:03:40 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item b6036204-7a8f-511b-8e0e-20a08deb58b8 sending to next pipeline
2025-12-05 20:03:40 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: b6036204-7a8f-511b-8e0e-20a08deb58b8, identifier: 1789086
2025-12-05 20:03:40 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/179/3a615519dd044728a7019fef65772a61/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:40 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/554/a1ddbd2fb11545b29f8f09d3ecd02c93/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:40 [ayvens.es] INFO: Found listing with ID: 1789096
2025-12-05 20:03:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789096%27 HTTP/1.1" 200 None
2025-12-05 20:03:40 [ayvens.es] INFO: Scrape type for 1789096: 1
2025-12-05 20:03:40 [ayvens.es] INFO: Extracting data for car ID: 1789096
2025-12-05 20:03:40 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 9ac1156a-1687-5086-9eb1-79880f7fdd9a with identifier: 1789096
2025-12-05 20:03:40 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9ac1156a-1687-5086-9eb1-79880f7fdd9a with scrape type 1
2025-12-05 20:03:40 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 9ac1156a-1687-5086-9eb1-79880f7fdd9a
2025-12-05 20:03:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F078%252F2ad5accff56c4be79007f77959cce947%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:40 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/078/2ad5accff56c4be79007f77959cce947/images/large/1.jpg not downloaded yet
2025-12-05 20:03:40 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: b6036204-7a8f-511b-8e0e-20a08deb58b8 with identifier: 1789086
2025-12-05 20:03:40 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:03:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:03:40 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: b6036204-7a8f-511b-8e0e-20a08deb58b8 with identifier: 1789086
2025-12-05 20:03:40 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: b6036204-7a8f-511b-8e0e-20a08deb58b8 with identifier: 1789086 to the API
2025-12-05 20:03:40 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:40 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:40 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/760/e595fb00fc094e88b2a9de66e06c300d/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 124,
 'color': 'Blanco alpino',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.6L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'b6036204-7a8f-511b-8e0e-20a08deb58b8',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'OPEL',
 'model': 'INSIGNIA',
 'odometer': 107711,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_b6036204-7a8f-511b-8e0e-20a08deb58b8/b6036204-7a8f-511b-8e0e-20a08deb58b8_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 339336,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/760/e595fb00fc094e88b2a9de66e06c300d/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-10-01',
 'seats_number': 5,
 'seller_name': None,
 'title': 'OPEL INSIGNIA GS 1.6 CDTi 100kW Turbo D Selective',
 'transmission': 'manual',
 'trim': 'GS 1.6 CDTi 100kW Turbo D Selective',
 'vin': None,
 'year': 2019}
2025-12-05 20:03:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789086%27 HTTP/1.1" 200 None
2025-12-05 20:03:40 [ayvens.es] INFO: Saving data for 1789086: {'created_time': 1764965020.746833, 'last_price_update_time': 1764965020.746845, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789086') HTTP/1.1" 204 0
2025-12-05 20:03:40 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:40 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:40 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:40 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0c762d0>
2025-12-05 20:03:40 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:40 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:41 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1969
2025-12-05 20:03:41 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:41 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:41 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:41 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_c9030dd5-8142-58e5-b54c-5f4202d63b21/c9030dd5-8142-58e5-b54c-5f4202d63b21_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F594%252F322c958ac29d4d7f86d13bcb174ffe76%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:41 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/594/322c958ac29d4d7f86d13bcb174ffe76/images/large/1.jpg saved to cache
2025-12-05 20:03:41 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item c9030dd5-8142-58e5-b54c-5f4202d63b21
2025-12-05 20:03:41 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: c9030dd5-8142-58e5-b54c-5f4202d63b21, skipping ID generation
2025-12-05 20:03:41 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item c9030dd5-8142-58e5-b54c-5f4202d63b21 with scrape type 1
2025-12-05 20:03:41 [crawlers.middlewares.photo_download_middleware] INFO: Removing item c9030dd5-8142-58e5-b54c-5f4202d63b21 from tracking dictionaries
2025-12-05 20:03:41 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item c9030dd5-8142-58e5-b54c-5f4202d63b21 sending to next pipeline
2025-12-05 20:03:41 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: c9030dd5-8142-58e5-b54c-5f4202d63b21, identifier: 1789088
2025-12-05 20:03:41 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:41 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:41 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:41 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b043b9d0>
2025-12-05 20:03:41 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:41 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:41 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:03:41 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:41 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:41 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:41 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_cee3693d-47c1-54d4-912f-503f4dbe9a77/cee3693d-47c1-54d4-912f-503f4dbe9a77_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F534%252Fce2f994fcd7f4babba058cfad877d5d9%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:41 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/534/ce2f994fcd7f4babba058cfad877d5d9/images/large/1.jpg saved to cache
2025-12-05 20:03:41 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item cee3693d-47c1-54d4-912f-503f4dbe9a77
2025-12-05 20:03:41 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: cee3693d-47c1-54d4-912f-503f4dbe9a77, skipping ID generation
2025-12-05 20:03:41 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item cee3693d-47c1-54d4-912f-503f4dbe9a77 with scrape type 1
2025-12-05 20:03:41 [crawlers.middlewares.photo_download_middleware] INFO: Removing item cee3693d-47c1-54d4-912f-503f4dbe9a77 from tracking dictionaries
2025-12-05 20:03:41 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item cee3693d-47c1-54d4-912f-503f4dbe9a77 sending to next pipeline
2025-12-05 20:03:41 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: cee3693d-47c1-54d4-912f-503f4dbe9a77, identifier: 1789087
2025-12-05 20:03:41 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:41 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:41 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:41 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b05235d0>
2025-12-05 20:03:41 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:41 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:41 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:03:41 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:42 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:42 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:42 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_a41a4c9b-2f22-5564-878f-c6d18bb49bce/a41a4c9b-2f22-5564-878f-c6d18bb49bce_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F656%252F525b650b54cf424b8d77095a86450987%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:42 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/656/525b650b54cf424b8d77095a86450987/images/large/1.jpg saved to cache
2025-12-05 20:03:42 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item a41a4c9b-2f22-5564-878f-c6d18bb49bce
2025-12-05 20:03:42 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: a41a4c9b-2f22-5564-878f-c6d18bb49bce, skipping ID generation
2025-12-05 20:03:42 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a41a4c9b-2f22-5564-878f-c6d18bb49bce with scrape type 1
2025-12-05 20:03:42 [crawlers.middlewares.photo_download_middleware] INFO: Removing item a41a4c9b-2f22-5564-878f-c6d18bb49bce from tracking dictionaries
2025-12-05 20:03:42 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item a41a4c9b-2f22-5564-878f-c6d18bb49bce sending to next pipeline
2025-12-05 20:03:42 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: a41a4c9b-2f22-5564-878f-c6d18bb49bce, identifier: 1789089
2025-12-05 20:03:42 [ayvens.es] INFO: Found listing with ID: 1789097
2025-12-05 20:03:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789097%27 HTTP/1.1" 200 None
2025-12-05 20:03:42 [ayvens.es] INFO: Scrape type for 1789097: 1
2025-12-05 20:03:42 [ayvens.es] INFO: Extracting data for car ID: 1789097
2025-12-05 20:03:42 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 059d6b12-b4d0-585b-80c7-e6c7cb3a3ebb with identifier: 1789097
2025-12-05 20:03:42 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 059d6b12-b4d0-585b-80c7-e6c7cb3a3ebb with scrape type 1
2025-12-05 20:03:42 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 059d6b12-b4d0-585b-80c7-e6c7cb3a3ebb
2025-12-05 20:03:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F833%252F96517236121d45de8d07fe3c883e2477%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:42 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/833/96517236121d45de8d07fe3c883e2477/images/large/1.jpg not downloaded yet
2025-12-05 20:03:42 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: c9030dd5-8142-58e5-b54c-5f4202d63b21 with identifier: 1789088
2025-12-05 20:03:42 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: c9030dd5-8142-58e5-b54c-5f4202d63b21 with identifier: 1789088
2025-12-05 20:03:42 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: c9030dd5-8142-58e5-b54c-5f4202d63b21 with identifier: 1789088 to the API
2025-12-05 20:03:42 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:42 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:42 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/594/322c958ac29d4d7f86d13bcb174ffe76/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 98,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.6L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'c9030dd5-8142-58e5-b54c-5f4202d63b21',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'FIAT',
 'model': 'TIPO',
 'odometer': 87948,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_c9030dd5-8142-58e5-b54c-5f4202d63b21/c9030dd5-8142-58e5-b54c-5f4202d63b21_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 346021,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/594/322c958ac29d4d7f86d13bcb174ffe76/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2018-10-04',
 'seats_number': 5,
 'seller_name': None,
 'title': 'FIAT TIPO (E) 1.6 16v Business 120cv Multijet II',
 'transmission': 'manual',
 'trim': '(E) 1.6 16v Business 120cv Multijet II',
 'vin': None,
 'year': 2018}
2025-12-05 20:03:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789088%27 HTTP/1.1" 200 None
2025-12-05 20:03:42 [ayvens.es] INFO: Saving data for 1789088: {'created_time': 1764965022.656872, 'last_price_update_time': 1764965022.656882, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789088') HTTP/1.1" 204 0
2025-12-05 20:03:42 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: cee3693d-47c1-54d4-912f-503f4dbe9a77 with identifier: 1789087
2025-12-05 20:03:42 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: cee3693d-47c1-54d4-912f-503f4dbe9a77 with identifier: 1789087
2025-12-05 20:03:42 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: cee3693d-47c1-54d4-912f-503f4dbe9a77 with identifier: 1789087 to the API
2025-12-05 20:03:42 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:42 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:43 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/534/ce2f994fcd7f4babba058cfad877d5d9/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 106,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'cee3693d-47c1-54d4-912f-503f4dbe9a77',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': '208',
 'odometer': 76474,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_cee3693d-47c1-54d4-912f-503f4dbe9a77/cee3693d-47c1-54d4-912f-503f4dbe9a77_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 332549,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/534/ce2f994fcd7f4babba058cfad877d5d9/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-05-26',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT 208 BlueHDi 73kW (100CV) Active',
 'transmission': 'manual',
 'trim': 'BlueHDi 73kW (100CV) Active',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789087%27 HTTP/1.1" 200 None
2025-12-05 20:03:43 [ayvens.es] INFO: Saving data for 1789087: {'created_time': 1764965023.098682, 'last_price_update_time': 1764965023.098693, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789087') HTTP/1.1" 204 0
2025-12-05 20:03:43 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: a41a4c9b-2f22-5564-878f-c6d18bb49bce with identifier: 1789089
2025-12-05 20:03:43 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:03:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:03:43 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: a41a4c9b-2f22-5564-878f-c6d18bb49bce with identifier: 1789089
2025-12-05 20:03:43 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: a41a4c9b-2f22-5564-878f-c6d18bb49bce with identifier: 1789089 to the API
2025-12-05 20:03:43 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:43 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:43 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/656/525b650b54cf424b8d77095a86450987/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 106,
 'color': 'Blanco glaciar',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'a41a4c9b-2f22-5564-878f-c6d18bb49bce',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'CLIO',
 'odometer': 27224,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_a41a4c9b-2f22-5564-878f-c6d18bb49bce/a41a4c9b-2f22-5564-878f-c6d18bb49bce_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 354121,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/656/525b650b54cf424b8d77095a86450987/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-07-12',
 'seats_number': 5,
 'seller_name': None,
 'title': 'RENAULT CLIO Business dCi 66kW (90CV) -18',
 'transmission': 'manual',
 'trim': 'Business dCi 66kW (90CV) -18',
 'vin': None,
 'year': 2019}
2025-12-05 20:03:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789089%27 HTTP/1.1" 200 None
2025-12-05 20:03:43 [ayvens.es] INFO: Saving data for 1789089: {'created_time': 1764965023.595374, 'last_price_update_time': 1764965023.595386, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789089') HTTP/1.1" 204 0
2025-12-05 20:03:43 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:43 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:43 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:43 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b047ae50>
2025-12-05 20:03:43 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:43 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:43 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1969
2025-12-05 20:03:43 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:43 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:43 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:43 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_55c967be-cc84-53df-92dc-6f3267fef603/55c967be-cc84-53df-92dc-6f3267fef603_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F179%252F3a615519dd044728a7019fef65772a61%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:43 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/179/3a615519dd044728a7019fef65772a61/images/large/1.jpg saved to cache
2025-12-05 20:03:43 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 55c967be-cc84-53df-92dc-6f3267fef603
2025-12-05 20:03:43 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 55c967be-cc84-53df-92dc-6f3267fef603, skipping ID generation
2025-12-05 20:03:43 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 55c967be-cc84-53df-92dc-6f3267fef603 with scrape type 1
2025-12-05 20:03:43 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 55c967be-cc84-53df-92dc-6f3267fef603 from tracking dictionaries
2025-12-05 20:03:43 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 55c967be-cc84-53df-92dc-6f3267fef603 sending to next pipeline
2025-12-05 20:03:44 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 55c967be-cc84-53df-92dc-6f3267fef603, identifier: 1789090
2025-12-05 20:03:44 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:44 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:44 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:44 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0ae7310>
2025-12-05 20:03:44 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:44 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:44 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:03:44 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:44 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:44 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:44 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_f1f8c897-6845-5f95-8b7a-229e8a6c2798/f1f8c897-6845-5f95-8b7a-229e8a6c2798_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F554%252Fa1ddbd2fb11545b29f8f09d3ecd02c93%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:44 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/554/a1ddbd2fb11545b29f8f09d3ecd02c93/images/large/1.jpg saved to cache
2025-12-05 20:03:44 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item f1f8c897-6845-5f95-8b7a-229e8a6c2798
2025-12-05 20:03:44 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: f1f8c897-6845-5f95-8b7a-229e8a6c2798, skipping ID generation
2025-12-05 20:03:44 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f1f8c897-6845-5f95-8b7a-229e8a6c2798 with scrape type 1
2025-12-05 20:03:44 [crawlers.middlewares.photo_download_middleware] INFO: Removing item f1f8c897-6845-5f95-8b7a-229e8a6c2798 from tracking dictionaries
2025-12-05 20:03:44 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item f1f8c897-6845-5f95-8b7a-229e8a6c2798 sending to next pipeline
2025-12-05 20:03:44 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: f1f8c897-6845-5f95-8b7a-229e8a6c2798, identifier: 1789091
2025-12-05 20:03:44 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/507/17428490c4bb446eb2e65667dbf95cce/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:44 [ayvens.es] INFO: Found listing with ID: 1789098
2025-12-05 20:03:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789098%27 HTTP/1.1" 200 None
2025-12-05 20:03:44 [ayvens.es] INFO: Scrape type for 1789098: 1
2025-12-05 20:03:44 [ayvens.es] INFO: Extracting data for car ID: 1789098
2025-12-05 20:03:44 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: ddfce582-77c4-561c-beda-aaf25b6780b9 with identifier: 1789098
2025-12-05 20:03:44 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ddfce582-77c4-561c-beda-aaf25b6780b9 with scrape type 1
2025-12-05 20:03:44 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item ddfce582-77c4-561c-beda-aaf25b6780b9
2025-12-05 20:03:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F074%252F757cffe7207f4f14bfc328b4e48c14d0%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:44 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/074/757cffe7207f4f14bfc328b4e48c14d0/images/large/1.jpg not downloaded yet
2025-12-05 20:03:44 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 55c967be-cc84-53df-92dc-6f3267fef603 with identifier: 1789090
2025-12-05 20:03:44 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 55c967be-cc84-53df-92dc-6f3267fef603 with identifier: 1789090
2025-12-05 20:03:44 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 55c967be-cc84-53df-92dc-6f3267fef603 with identifier: 1789090 to the API
2025-12-05 20:03:44 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:44 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:44 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/179/3a615519dd044728a7019fef65772a61/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 106,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '55c967be-cc84-53df-92dc-6f3267fef603',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'PEUGEOT',
 'model': '208',
 'odometer': 17503,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_55c967be-cc84-53df-92dc-6f3267fef603/55c967be-cc84-53df-92dc-6f3267fef603_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 363573,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/179/3a615519dd044728a7019fef65772a61/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-09-09',
 'seats_number': 5,
 'seller_name': None,
 'title': 'PEUGEOT 208 BlueHDi 73kW (100CV) Active',
 'transmission': 'manual',
 'trim': 'BlueHDi 73kW (100CV) Active',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789090%27 HTTP/1.1" 200 None
2025-12-05 20:03:44 [ayvens.es] INFO: Saving data for 1789090: {'created_time': 1764965024.933402, 'last_price_update_time': 1764965024.933414, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:44 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789090') HTTP/1.1" 204 0
2025-12-05 20:03:44 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: f1f8c897-6845-5f95-8b7a-229e8a6c2798 with identifier: 1789091
2025-12-05 20:03:44 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: f1f8c897-6845-5f95-8b7a-229e8a6c2798 with identifier: 1789091
2025-12-05 20:03:45 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: f1f8c897-6845-5f95-8b7a-229e8a6c2798 with identifier: 1789091 to the API
2025-12-05 20:03:45 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:45 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:45 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/554/a1ddbd2fb11545b29f8f09d3ecd02c93/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 109,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': 'f1f8c897-6845-5f95-8b7a-229e8a6c2798',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'GOLF',
 'odometer': 111460,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_f1f8c897-6845-5f95-8b7a-229e8a6c2798/f1f8c897-6845-5f95-8b7a-229e8a6c2798_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 318079,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/554/a1ddbd2fb11545b29f8f09d3ecd02c93/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-08-29',
 'seats_number': 5,
 'seller_name': None,
 'title': 'VOLKSWAGEN GOLF Ready2Go 1.0 TSI 85kW (115CV)',
 'transmission': 'manual',
 'trim': 'Ready2Go 1.0 TSI 85kW (115CV)',
 'vin': None,
 'year': 2019}
2025-12-05 20:03:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789091%27 HTTP/1.1" 200 None
2025-12-05 20:03:45 [ayvens.es] INFO: Saving data for 1789091: {'created_time': 1764965025.330591, 'last_price_update_time': 1764965025.3306, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789091') HTTP/1.1" 204 0
2025-12-05 20:03:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/609/5c98d376f18f44efb9ebba4bbf1bd8b5/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:45 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/464/003f485f8f704b729c9637118d40e078/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:45 [ayvens.es] INFO: Found listing with ID: 1789099
2025-12-05 20:03:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789099%27 HTTP/1.1" 200 None
2025-12-05 20:03:45 [ayvens.es] INFO: Scrape type for 1789099: 1
2025-12-05 20:03:45 [ayvens.es] INFO: Extracting data for car ID: 1789099
2025-12-05 20:03:45 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: e30cb6ed-515f-505a-8146-ddd3d35da43d with identifier: 1789099
2025-12-05 20:03:45 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e30cb6ed-515f-505a-8146-ddd3d35da43d with scrape type 1
2025-12-05 20:03:45 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item e30cb6ed-515f-505a-8146-ddd3d35da43d
2025-12-05 20:03:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F215%252Ffb5392df3a3240ccbbd19a9b551a9fb8%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:45 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/215/fb5392df3a3240ccbbd19a9b551a9fb8/images/large/1.jpg not downloaded yet
2025-12-05 20:03:45 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:45 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:45 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:45 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0540b10>
2025-12-05 20:03:45 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:45 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:45 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:03:45 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:45 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:45 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:45 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_b70ecdd8-948b-5dce-8278-e86ef5ccf632/b70ecdd8-948b-5dce-8278-e86ef5ccf632_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F507%252F17428490c4bb446eb2e65667dbf95cce%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:45 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/507/17428490c4bb446eb2e65667dbf95cce/images/large/1.jpg saved to cache
2025-12-05 20:03:45 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item b70ecdd8-948b-5dce-8278-e86ef5ccf632
2025-12-05 20:03:45 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: b70ecdd8-948b-5dce-8278-e86ef5ccf632, skipping ID generation
2025-12-05 20:03:45 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b70ecdd8-948b-5dce-8278-e86ef5ccf632 with scrape type 1
2025-12-05 20:03:45 [crawlers.middlewares.photo_download_middleware] INFO: Removing item b70ecdd8-948b-5dce-8278-e86ef5ccf632 from tracking dictionaries
2025-12-05 20:03:45 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item b70ecdd8-948b-5dce-8278-e86ef5ccf632 sending to next pipeline
2025-12-05 20:03:45 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: b70ecdd8-948b-5dce-8278-e86ef5ccf632, identifier: 1789092
2025-12-05 20:03:45 [ayvens.es] INFO: Found listing with ID: 1789100
2025-12-05 20:03:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789100%27 HTTP/1.1" 200 None
2025-12-05 20:03:45 [ayvens.es] INFO: Scrape type for 1789100: 1
2025-12-05 20:03:45 [ayvens.es] INFO: Extracting data for car ID: 1789100
2025-12-05 20:03:45 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: df32eb45-d642-505a-be68-e7f6cc7ac133 with identifier: 1789100
2025-12-05 20:03:45 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item df32eb45-d642-505a-be68-e7f6cc7ac133 with scrape type 1
2025-12-05 20:03:45 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item df32eb45-d642-505a-be68-e7f6cc7ac133
2025-12-05 20:03:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:45 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F116%252F50186ef71afd4f798ae778b96f1baee0%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:45 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/116/50186ef71afd4f798ae778b96f1baee0/images/large/1.jpg not downloaded yet
2025-12-05 20:03:45 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: b70ecdd8-948b-5dce-8278-e86ef5ccf632 with identifier: 1789092
2025-12-05 20:03:45 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: b70ecdd8-948b-5dce-8278-e86ef5ccf632 with identifier: 1789092
2025-12-05 20:03:45 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: b70ecdd8-948b-5dce-8278-e86ef5ccf632 with identifier: 1789092 to the API
2025-12-05 20:03:45 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:46 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:46 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/507/17428490c4bb446eb2e65667dbf95cce/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 94,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'b70ecdd8-948b-5dce-8278-e86ef5ccf632',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'RENAULT',
 'model': 'CLIO',
 'odometer': 116793,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_b70ecdd8-948b-5dce-8278-e86ef5ccf632/b70ecdd8-948b-5dce-8278-e86ef5ccf632_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 347274,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/507/17428490c4bb446eb2e65667dbf95cce/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-01-25',
 'seats_number': 5,
 'seller_name': None,
 'title': 'RENAULT CLIO (O) Business Blue dCi 63 kW (85CV)',
 'transmission': 'manual',
 'trim': '(O) Business Blue dCi 63 kW (85CV)',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789092%27 HTTP/1.1" 200 None
2025-12-05 20:03:46 [ayvens.es] INFO: Saving data for 1789092: {'created_time': 1764965026.279141, 'last_price_update_time': 1764965026.279152, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789092') HTTP/1.1" 204 0
2025-12-05 20:03:46 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/851/372f90268e51412bae0483c8d4feff15/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:46 [ayvens.es] INFO: Found listing with ID: 1789101
2025-12-05 20:03:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789101%27 HTTP/1.1" 200 None
2025-12-05 20:03:46 [ayvens.es] INFO: Scrape type for 1789101: 1
2025-12-05 20:03:46 [ayvens.es] INFO: Extracting data for car ID: 1789101
2025-12-05 20:03:46 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: c350f53b-6cd1-5441-acf9-4b9c13e8fb06 with identifier: 1789101
2025-12-05 20:03:46 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item c350f53b-6cd1-5441-acf9-4b9c13e8fb06 with scrape type 1
2025-12-05 20:03:46 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item c350f53b-6cd1-5441-acf9-4b9c13e8fb06
2025-12-05 20:03:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F880%252F059bbe3af11d4fe7929af61236492554%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:46 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/880/059bbe3af11d4fe7929af61236492554/images/large/1.jpg not downloaded yet
2025-12-05 20:03:46 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/078/2ad5accff56c4be79007f77959cce947/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:46 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:46 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:46 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:46 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0acc090>
2025-12-05 20:03:46 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:46 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:46 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1963
2025-12-05 20:03:46 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:46 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:46 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:46 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_a43cf8e1-fd1b-5a35-ae8a-ac8bfc6449f2/a43cf8e1-fd1b-5a35-ae8a-ac8bfc6449f2_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:46 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F609%252F5c98d376f18f44efb9ebba4bbf1bd8b5%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:46 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/609/5c98d376f18f44efb9ebba4bbf1bd8b5/images/large/1.jpg saved to cache
2025-12-05 20:03:46 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item a43cf8e1-fd1b-5a35-ae8a-ac8bfc6449f2
2025-12-05 20:03:46 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: a43cf8e1-fd1b-5a35-ae8a-ac8bfc6449f2, skipping ID generation
2025-12-05 20:03:46 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a43cf8e1-fd1b-5a35-ae8a-ac8bfc6449f2 with scrape type 1
2025-12-05 20:03:46 [crawlers.middlewares.photo_download_middleware] INFO: Removing item a43cf8e1-fd1b-5a35-ae8a-ac8bfc6449f2 from tracking dictionaries
2025-12-05 20:03:46 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item a43cf8e1-fd1b-5a35-ae8a-ac8bfc6449f2 sending to next pipeline
2025-12-05 20:03:46 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: a43cf8e1-fd1b-5a35-ae8a-ac8bfc6449f2, identifier: 1789093
2025-12-05 20:03:46 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:46 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:46 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:46 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0452090>
2025-12-05 20:03:46 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:46 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:47 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:03:47 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:47 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:47 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:47 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_44cf5a86-4384-5f1e-a539-acb64af804af/44cf5a86-4384-5f1e-a539-acb64af804af_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F464%252F003f485f8f704b729c9637118d40e078%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:47 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/464/003f485f8f704b729c9637118d40e078/images/large/1.jpg saved to cache
2025-12-05 20:03:47 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 44cf5a86-4384-5f1e-a539-acb64af804af
2025-12-05 20:03:47 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 44cf5a86-4384-5f1e-a539-acb64af804af, skipping ID generation
2025-12-05 20:03:47 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 44cf5a86-4384-5f1e-a539-acb64af804af with scrape type 1
2025-12-05 20:03:47 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 44cf5a86-4384-5f1e-a539-acb64af804af from tracking dictionaries
2025-12-05 20:03:47 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 44cf5a86-4384-5f1e-a539-acb64af804af sending to next pipeline
2025-12-05 20:03:47 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 44cf5a86-4384-5f1e-a539-acb64af804af, identifier: 1789094
2025-12-05 20:03:47 [ayvens.es] INFO: Found listing with ID: 1789102
2025-12-05 20:03:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789102%27 HTTP/1.1" 200 None
2025-12-05 20:03:47 [ayvens.es] INFO: Scrape type for 1789102: 1
2025-12-05 20:03:47 [ayvens.es] INFO: Extracting data for car ID: 1789102
2025-12-05 20:03:47 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 0ea10ae5-c329-548e-a5c9-4031c5f1e75d with identifier: 1789102
2025-12-05 20:03:47 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 0ea10ae5-c329-548e-a5c9-4031c5f1e75d with scrape type 1
2025-12-05 20:03:47 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 0ea10ae5-c329-548e-a5c9-4031c5f1e75d
2025-12-05 20:03:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F375%252Fcc0e4ffc0abd4a739b641957a1fd25b2%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:47 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/375/cc0e4ffc0abd4a739b641957a1fd25b2/images/large/1.jpg not downloaded yet
2025-12-05 20:03:47 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: a43cf8e1-fd1b-5a35-ae8a-ac8bfc6449f2 with identifier: 1789093
2025-12-05 20:03:47 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:03:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:03:47 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: a43cf8e1-fd1b-5a35-ae8a-ac8bfc6449f2 with identifier: 1789093
2025-12-05 20:03:47 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: a43cf8e1-fd1b-5a35-ae8a-ac8bfc6449f2 with identifier: 1789093 to the API
2025-12-05 20:03:47 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:47 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/609/5c98d376f18f44efb9ebba4bbf1bd8b5/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 142,
 'color': 'Gris indy',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': 'a43cf8e1-fd1b-5a35-ae8a-ac8bfc6449f2',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'VOLKSWAGEN',
 'model': 'T-ROC',
 'odometer': 57293,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_a43cf8e1-fd1b-5a35-ae8a-ac8bfc6449f2/a43cf8e1-fd1b-5a35-ae8a-ac8bfc6449f2_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 375817,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/609/5c98d376f18f44efb9ebba4bbf1bd8b5/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2022-09-16',
 'seats_number': 5,
 'seller_name': None,
 'title': 'VOLKSWAGEN T-ROC Life 1.5 TSI 110kW (150CV) DSG',
 'transmission': 'manual',
 'trim': 'Life 1.5 TSI 110kW (150CV) DSG',
 'vin': None,
 'year': 2022}
2025-12-05 20:03:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789093%27 HTTP/1.1" 200 None
2025-12-05 20:03:47 [ayvens.es] INFO: Saving data for 1789093: {'created_time': 1764965027.82247, 'last_price_update_time': 1764965027.822481, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:47 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789093') HTTP/1.1" 204 0
2025-12-05 20:03:47 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 44cf5a86-4384-5f1e-a539-acb64af804af with identifier: 1789094
2025-12-05 20:03:47 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 44cf5a86-4384-5f1e-a539-acb64af804af with identifier: 1789094
2025-12-05 20:03:47 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 44cf5a86-4384-5f1e-a539-acb64af804af with identifier: 1789094 to the API
2025-12-05 20:03:47 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:48 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:48 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/464/003f485f8f704b729c9637118d40e078/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 111,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.8L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'hybrid',
 'id': '44cf5a86-4384-5f1e-a539-acb64af804af',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'TOYOTA',
 'model': 'C-HR',
 'odometer': 43468,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_44cf5a86-4384-5f1e-a539-acb64af804af/44cf5a86-4384-5f1e-a539-acb64af804af_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 324262,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/464/003f485f8f704b729c9637118d40e078/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-05-07',
 'seats_number': 5,
 'seller_name': None,
 'title': 'TOYOTA C-HR 1.8 125H Active',
 'transmission': 'manual',
 'trim': '1.8 125H Active',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789094%27 HTTP/1.1" 200 None
2025-12-05 20:03:48 [ayvens.es] INFO: Saving data for 1789094: {'created_time': 1764965028.229115, 'last_price_update_time': 1764965028.229126, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789094') HTTP/1.1" 204 0
2025-12-05 20:03:48 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/833/96517236121d45de8d07fe3c883e2477/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:48 [ayvens.es] INFO: Found listing with ID: 1789103
2025-12-05 20:03:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789103%27 HTTP/1.1" 200 None
2025-12-05 20:03:48 [ayvens.es] INFO: Scrape type for 1789103: 1
2025-12-05 20:03:48 [ayvens.es] INFO: Extracting data for car ID: 1789103
2025-12-05 20:03:48 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 83944ffd-c45c-555a-a77d-699661e66cb1 with identifier: 1789103
2025-12-05 20:03:48 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 83944ffd-c45c-555a-a77d-699661e66cb1 with scrape type 1
2025-12-05 20:03:48 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 83944ffd-c45c-555a-a77d-699661e66cb1
2025-12-05 20:03:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F771%252Fa98fb86df07440c197d9d4b07192419d%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:48 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/771/a98fb86df07440c197d9d4b07192419d/images/large/1.jpg not downloaded yet
2025-12-05 20:03:48 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:48 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:48 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:48 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0432090>
2025-12-05 20:03:48 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:48 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:48 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:03:48 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:48 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:48 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:48 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_ff3549f8-543f-5a48-9497-3ca212b2bd28/ff3549f8-543f-5a48-9497-3ca212b2bd28_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:48 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F851%252F372f90268e51412bae0483c8d4feff15%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:48 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/851/372f90268e51412bae0483c8d4feff15/images/large/1.jpg saved to cache
2025-12-05 20:03:48 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item ff3549f8-543f-5a48-9497-3ca212b2bd28
2025-12-05 20:03:48 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: ff3549f8-543f-5a48-9497-3ca212b2bd28, skipping ID generation
2025-12-05 20:03:48 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ff3549f8-543f-5a48-9497-3ca212b2bd28 with scrape type 1
2025-12-05 20:03:48 [crawlers.middlewares.photo_download_middleware] INFO: Removing item ff3549f8-543f-5a48-9497-3ca212b2bd28 from tracking dictionaries
2025-12-05 20:03:48 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item ff3549f8-543f-5a48-9497-3ca212b2bd28 sending to next pipeline
2025-12-05 20:03:48 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: ff3549f8-543f-5a48-9497-3ca212b2bd28, identifier: 1789095
2025-12-05 20:03:48 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:48 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:48 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:48 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b06996d0>
2025-12-05 20:03:48 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:48 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:48 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:03:48 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:49 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:49 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:49 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_9ac1156a-1687-5086-9eb1-79880f7fdd9a/9ac1156a-1687-5086-9eb1-79880f7fdd9a_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F078%252F2ad5accff56c4be79007f77959cce947%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:49 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/078/2ad5accff56c4be79007f77959cce947/images/large/1.jpg saved to cache
2025-12-05 20:03:49 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 9ac1156a-1687-5086-9eb1-79880f7fdd9a
2025-12-05 20:03:49 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 9ac1156a-1687-5086-9eb1-79880f7fdd9a, skipping ID generation
2025-12-05 20:03:49 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9ac1156a-1687-5086-9eb1-79880f7fdd9a with scrape type 1
2025-12-05 20:03:49 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 9ac1156a-1687-5086-9eb1-79880f7fdd9a from tracking dictionaries
2025-12-05 20:03:49 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 9ac1156a-1687-5086-9eb1-79880f7fdd9a sending to next pipeline
2025-12-05 20:03:49 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 9ac1156a-1687-5086-9eb1-79880f7fdd9a, identifier: 1789096
2025-12-05 20:03:49 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: ff3549f8-543f-5a48-9497-3ca212b2bd28 with identifier: 1789095
2025-12-05 20:03:49 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: ff3549f8-543f-5a48-9497-3ca212b2bd28 with identifier: 1789095
2025-12-05 20:03:49 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: ff3549f8-543f-5a48-9497-3ca212b2bd28 with identifier: 1789095 to the API
2025-12-05 20:03:49 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:49 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:49 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/851/372f90268e51412bae0483c8d4feff15/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 108,
 'color': 'Saphirschwarz',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'ff3549f8-543f-5a48-9497-3ca212b2bd28',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'BMW',
 'model': 'SERIES 1',
 'odometer': 106643,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ff3549f8-543f-5a48-9497-3ca212b2bd28/ff3549f8-543f-5a48-9497-3ca212b2bd28_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 391803,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/851/372f90268e51412bae0483c8d4feff15/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-10-28',
 'seats_number': 5,
 'seller_name': None,
 'title': 'BMW SERIES 1 118d',
 'transmission': 'manual',
 'trim': '118d',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789095%27 HTTP/1.1" 200 None
2025-12-05 20:03:49 [ayvens.es] INFO: Saving data for 1789095: {'created_time': 1764965029.63339, 'last_price_update_time': 1764965029.633398, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:49 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789095') HTTP/1.1" 204 0
2025-12-05 20:03:49 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 9ac1156a-1687-5086-9eb1-79880f7fdd9a with identifier: 1789096
2025-12-05 20:03:49 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 9ac1156a-1687-5086-9eb1-79880f7fdd9a with identifier: 1789096
2025-12-05 20:03:49 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 9ac1156a-1687-5086-9eb1-79880f7fdd9a with identifier: 1789096 to the API
2025-12-05 20:03:49 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:50 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:50 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/078/2ad5accff56c4be79007f77959cce947/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 125,
 'color': 'Negro',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': '9ac1156a-1687-5086-9eb1-79880f7fdd9a',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'KIA',
 'model': 'STONIC',
 'odometer': 25810,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_9ac1156a-1687-5086-9eb1-79880f7fdd9a/9ac1156a-1687-5086-9eb1-79880f7fdd9a_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 382757,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/078/2ad5accff56c4be79007f77959cce947/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-09-28',
 'seats_number': 5,
 'seller_name': None,
 'title': 'KIA STONIC 1.0 T-GDi 88kW (120CV) MHEV Drive DCT',
 'transmission': 'manual',
 'trim': '1.0 T-GDi 88kW (120CV) MHEV Drive DCT',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789096%27 HTTP/1.1" 200 None
2025-12-05 20:03:50 [ayvens.es] INFO: Saving data for 1789096: {'created_time': 1764965030.15765, 'last_price_update_time': 1764965030.157661, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789096') HTTP/1.1" 204 0
2025-12-05 20:03:50 [ayvens.es] INFO: Found listing with ID: 1789104
2025-12-05 20:03:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789104%27 HTTP/1.1" 200 None
2025-12-05 20:03:50 [ayvens.es] INFO: Scrape type for 1789104: 1
2025-12-05 20:03:50 [ayvens.es] INFO: Extracting data for car ID: 1789104
2025-12-05 20:03:50 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 42a795cb-5341-5e0e-bfae-529bbbbe2830 with identifier: 1789104
2025-12-05 20:03:50 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 42a795cb-5341-5e0e-bfae-529bbbbe2830 with scrape type 1
2025-12-05 20:03:50 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 42a795cb-5341-5e0e-bfae-529bbbbe2830
2025-12-05 20:03:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F513%252Ff83615107da9413da4381ec4c20ae5ee%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:50 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/513/f83615107da9413da4381ec4c20ae5ee/images/large/1.jpg not downloaded yet
2025-12-05 20:03:50 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/215/fb5392df3a3240ccbbd19a9b551a9fb8/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:50 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:50 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:50 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:50 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0c7a950>
2025-12-05 20:03:50 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:50 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:50 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:03:50 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:50 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:50 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:50 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_059d6b12-b4d0-585b-80c7-e6c7cb3a3ebb/059d6b12-b4d0-585b-80c7-e6c7cb3a3ebb_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F833%252F96517236121d45de8d07fe3c883e2477%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:50 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/833/96517236121d45de8d07fe3c883e2477/images/large/1.jpg saved to cache
2025-12-05 20:03:50 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 059d6b12-b4d0-585b-80c7-e6c7cb3a3ebb
2025-12-05 20:03:50 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 059d6b12-b4d0-585b-80c7-e6c7cb3a3ebb, skipping ID generation
2025-12-05 20:03:50 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 059d6b12-b4d0-585b-80c7-e6c7cb3a3ebb with scrape type 1
2025-12-05 20:03:50 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 059d6b12-b4d0-585b-80c7-e6c7cb3a3ebb from tracking dictionaries
2025-12-05 20:03:50 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 059d6b12-b4d0-585b-80c7-e6c7cb3a3ebb sending to next pipeline
2025-12-05 20:03:50 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 059d6b12-b4d0-585b-80c7-e6c7cb3a3ebb, identifier: 1789097
2025-12-05 20:03:50 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/074/757cffe7207f4f14bfc328b4e48c14d0/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:50 [ayvens.es] INFO: Found listing with ID: 1789105
2025-12-05 20:03:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789105%27 HTTP/1.1" 200 None
2025-12-05 20:03:50 [ayvens.es] INFO: Scrape type for 1789105: 1
2025-12-05 20:03:50 [ayvens.es] INFO: Extracting data for car ID: 1789105
2025-12-05 20:03:50 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 92f550bf-afa8-554e-a076-5488bc8e638b with identifier: 1789105
2025-12-05 20:03:50 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 92f550bf-afa8-554e-a076-5488bc8e638b with scrape type 1
2025-12-05 20:03:50 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 92f550bf-afa8-554e-a076-5488bc8e638b
2025-12-05 20:03:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F197%252F8fa52c0613304399a1d776487b17a28f%252Fimages%252Flarge%252F1.jpg%27 HTTP/1.1" 200 None
2025-12-05 20:03:50 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/197/8fa52c0613304399a1d776487b17a28f/images/large/1.jpg not downloaded yet
2025-12-05 20:03:50 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 059d6b12-b4d0-585b-80c7-e6c7cb3a3ebb with identifier: 1789097
2025-12-05 20:03:50 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 059d6b12-b4d0-585b-80c7-e6c7cb3a3ebb with identifier: 1789097
2025-12-05 20:03:50 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'citroën' containing 'citroën' and model 'jumpy' containing 'jumpy'
2025-12-05 20:03:50 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 059d6b12-b4d0-585b-80c7-e6c7cb3a3ebb spider: ayvens.es
2025-12-05 20:03:51 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 059d6b12-b4d0-585b-80c7-e6c7cb3a3ebb due to rule: makes_models_not_allowed.json, error: Rejected due to make 'citroën' containing 'citroën' and model 'jumpy' containing 'jumpy'
{'auction_title': '',
 'c_o_2_emission_value': 163,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '059d6b12-b4d0-585b-80c7-e6c7cb3a3ebb',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'CITROËN',
 'model': 'JUMPY',
 'odometer': 132519,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_059d6b12-b4d0-585b-80c7-e6c7cb3a3ebb/059d6b12-b4d0-585b-80c7-e6c7cb3a3ebb_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 337864,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/833/96517236121d45de8d07fe3c883e2477/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-02-02',
 'seats_number': 2,
 'seller_name': None,
 'title': 'CITROËN JUMPY Talla M BlueHDi 120 S&S 6v Control',
 'transmission': 'manual',
 'trim': 'Talla M BlueHDi 120 S&S 6v Control',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789097%27 HTTP/1.1" 200 None
2025-12-05 20:03:51 [ayvens.es] INFO: Saving data for 1789097: {'created_time': 1764965031.078502, 'last_price_update_time': 1764965031.078514, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789097') HTTP/1.1" 204 0
2025-12-05 20:03:51 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/116/50186ef71afd4f798ae778b96f1baee0/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:51 [ayvens.es] INFO: Fetching listings with token: 3b2cc62fd26c4e29a762db3de181266b and skip: 300
2025-12-05 20:03:51 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/880/059bbe3af11d4fe7929af61236492554/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:51 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:51 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:51 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:51 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b9a990>
2025-12-05 20:03:51 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:51 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:51 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:03:51 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:51 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:51 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:51 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_e30cb6ed-515f-505a-8146-ddd3d35da43d/e30cb6ed-515f-505a-8146-ddd3d35da43d_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F215%252Ffb5392df3a3240ccbbd19a9b551a9fb8%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:51 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/215/fb5392df3a3240ccbbd19a9b551a9fb8/images/large/1.jpg saved to cache
2025-12-05 20:03:51 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item e30cb6ed-515f-505a-8146-ddd3d35da43d
2025-12-05 20:03:51 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: e30cb6ed-515f-505a-8146-ddd3d35da43d, skipping ID generation
2025-12-05 20:03:51 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e30cb6ed-515f-505a-8146-ddd3d35da43d with scrape type 1
2025-12-05 20:03:51 [crawlers.middlewares.photo_download_middleware] INFO: Removing item e30cb6ed-515f-505a-8146-ddd3d35da43d from tracking dictionaries
2025-12-05 20:03:51 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item e30cb6ed-515f-505a-8146-ddd3d35da43d sending to next pipeline
2025-12-05 20:03:51 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: e30cb6ed-515f-505a-8146-ddd3d35da43d, identifier: 1789099
2025-12-05 20:03:51 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:51 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:51 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:51 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0aa5c50>
2025-12-05 20:03:51 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:51 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:51 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1973
2025-12-05 20:03:51 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:51 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:51 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:51 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_ddfce582-77c4-561c-beda-aaf25b6780b9/ddfce582-77c4-561c-beda-aaf25b6780b9_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:51 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F074%252F757cffe7207f4f14bfc328b4e48c14d0%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:51 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/074/757cffe7207f4f14bfc328b4e48c14d0/images/large/1.jpg saved to cache
2025-12-05 20:03:51 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item ddfce582-77c4-561c-beda-aaf25b6780b9
2025-12-05 20:03:51 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: ddfce582-77c4-561c-beda-aaf25b6780b9, skipping ID generation
2025-12-05 20:03:51 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ddfce582-77c4-561c-beda-aaf25b6780b9 with scrape type 1
2025-12-05 20:03:51 [crawlers.middlewares.photo_download_middleware] INFO: Removing item ddfce582-77c4-561c-beda-aaf25b6780b9 from tracking dictionaries
2025-12-05 20:03:51 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item ddfce582-77c4-561c-beda-aaf25b6780b9 sending to next pipeline
2025-12-05 20:03:51 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: ddfce582-77c4-561c-beda-aaf25b6780b9, identifier: 1789098
2025-12-05 20:03:51 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: e30cb6ed-515f-505a-8146-ddd3d35da43d with identifier: 1789099
2025-12-05 20:03:51 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: e30cb6ed-515f-505a-8146-ddd3d35da43d with identifier: 1789099
2025-12-05 20:03:52 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'opel' containing 'opel' and model 'vivaro' containing 'vivaro'
2025-12-05 20:03:52 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: e30cb6ed-515f-505a-8146-ddd3d35da43d spider: ayvens.es
2025-12-05 20:03:52 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: e30cb6ed-515f-505a-8146-ddd3d35da43d due to rule: makes_models_not_allowed.json, error: Rejected due to make 'opel' containing 'opel' and model 'vivaro' containing 'vivaro'
{'auction_title': '',
 'c_o_2_emission_value': 128,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'e30cb6ed-515f-505a-8146-ddd3d35da43d',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'OPEL',
 'model': 'VIVARO',
 'odometer': 148420,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_e30cb6ed-515f-505a-8146-ddd3d35da43d/e30cb6ed-515f-505a-8146-ddd3d35da43d_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 314718,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/215/fb5392df3a3240ccbbd19a9b551a9fb8/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-11-16',
 'seats_number': 2,
 'seller_name': None,
 'title': 'OPEL VIVARO 1.5 Diesel 74kW (100CV) M Std Express',
 'transmission': 'manual',
 'trim': '1.5 Diesel 74kW (100CV) M Std Express',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789099%27 HTTP/1.1" 200 None
2025-12-05 20:03:52 [ayvens.es] INFO: Saving data for 1789099: {'created_time': 1764965032.250588, 'last_price_update_time': 1764965032.250596, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789099') HTTP/1.1" 204 0
2025-12-05 20:03:52 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: ddfce582-77c4-561c-beda-aaf25b6780b9 with identifier: 1789098
2025-12-05 20:03:52 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: ddfce582-77c4-561c-beda-aaf25b6780b9 with identifier: 1789098
2025-12-05 20:03:52 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'citroën' containing 'citroën' and model 'jumpy' containing 'jumpy'
2025-12-05 20:03:52 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: ddfce582-77c4-561c-beda-aaf25b6780b9 spider: ayvens.es
2025-12-05 20:03:52 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: ddfce582-77c4-561c-beda-aaf25b6780b9 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'citroën' containing 'citroën' and model 'jumpy' containing 'jumpy'
{'auction_title': '',
 'c_o_2_emission_value': 188,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '2.0L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'ddfce582-77c4-561c-beda-aaf25b6780b9',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'CITROËN',
 'model': 'JUMPY',
 'odometer': 164484,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_ddfce582-77c4-561c-beda-aaf25b6780b9/ddfce582-77c4-561c-beda-aaf25b6780b9_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 332596,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/074/757cffe7207f4f14bfc328b4e48c14d0/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-02-03',
 'seats_number': 3,
 'seller_name': None,
 'title': 'CITROËN JUMPY Talla M BlueHDi 120 S&S 6v Club (CA)',
 'transmission': 'manual',
 'trim': 'Talla M BlueHDi 120 S&S 6v Club (CA)',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789098%27 HTTP/1.1" 200 None
2025-12-05 20:03:52 [ayvens.es] INFO: Saving data for 1789098: {'created_time': 1764965032.499792, 'last_price_update_time': 1764965032.499803, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789098') HTTP/1.1" 204 0
2025-12-05 20:03:52 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 1st time calling it.
2025-12-05 20:03:52 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/375/cc0e4ffc0abd4a739b641957a1fd25b2/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:52 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/771/a98fb86df07440c197d9d4b07192419d/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:52 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:52 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:52 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:52 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0b79bd0>
2025-12-05 20:03:52 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:52 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:52 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:03:52 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:52 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:52 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:52 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_df32eb45-d642-505a-be68-e7f6cc7ac133/df32eb45-d642-505a-be68-e7f6cc7ac133_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:52 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F116%252F50186ef71afd4f798ae778b96f1baee0%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:52 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/116/50186ef71afd4f798ae778b96f1baee0/images/large/1.jpg saved to cache
2025-12-05 20:03:52 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item df32eb45-d642-505a-be68-e7f6cc7ac133
2025-12-05 20:03:52 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: df32eb45-d642-505a-be68-e7f6cc7ac133, skipping ID generation
2025-12-05 20:03:52 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item df32eb45-d642-505a-be68-e7f6cc7ac133 with scrape type 1
2025-12-05 20:03:52 [crawlers.middlewares.photo_download_middleware] INFO: Removing item df32eb45-d642-505a-be68-e7f6cc7ac133 from tracking dictionaries
2025-12-05 20:03:52 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item df32eb45-d642-505a-be68-e7f6cc7ac133 sending to next pipeline
2025-12-05 20:03:53 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: df32eb45-d642-505a-be68-e7f6cc7ac133, identifier: 1789100
2025-12-05 20:03:53 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:53 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:53 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:53 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0456310>
2025-12-05 20:03:53 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:53 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:53 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1971
2025-12-05 20:03:53 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:53 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:53 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:53 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_c350f53b-6cd1-5441-acf9-4b9c13e8fb06/c350f53b-6cd1-5441-acf9-4b9c13e8fb06_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F880%252F059bbe3af11d4fe7929af61236492554%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:53 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/880/059bbe3af11d4fe7929af61236492554/images/large/1.jpg saved to cache
2025-12-05 20:03:53 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item c350f53b-6cd1-5441-acf9-4b9c13e8fb06
2025-12-05 20:03:53 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: c350f53b-6cd1-5441-acf9-4b9c13e8fb06, skipping ID generation
2025-12-05 20:03:53 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item c350f53b-6cd1-5441-acf9-4b9c13e8fb06 with scrape type 1
2025-12-05 20:03:53 [crawlers.middlewares.photo_download_middleware] INFO: Removing item c350f53b-6cd1-5441-acf9-4b9c13e8fb06 from tracking dictionaries
2025-12-05 20:03:53 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item c350f53b-6cd1-5441-acf9-4b9c13e8fb06 sending to next pipeline
2025-12-05 20:03:53 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: c350f53b-6cd1-5441-acf9-4b9c13e8fb06, identifier: 1789101
2025-12-05 20:03:53 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: df32eb45-d642-505a-be68-e7f6cc7ac133 with identifier: 1789100
2025-12-05 20:03:53 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2025-12-05 20:03:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='en',RowKey='en') HTTP/1.1" 204 0
2025-12-05 20:03:53 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: df32eb45-d642-505a-be68-e7f6cc7ac133 with identifier: 1789100
2025-12-05 20:03:53 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: df32eb45-d642-505a-be68-e7f6cc7ac133 with identifier: 1789100 to the API
2025-12-05 20:03:53 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:53 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:53 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/116/50186ef71afd4f798ae778b96f1baee0/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 114,
 'color': 'Blanco ion',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.2L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'petrol',
 'id': 'df32eb45-d642-505a-be68-e7f6cc7ac133',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'OPEL',
 'model': 'CORSA',
 'odometer': 29180,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_df32eb45-d642-505a-be68-e7f6cc7ac133/df32eb45-d642-505a-be68-e7f6cc7ac133_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 320472,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/116/50186ef71afd4f798ae778b96f1baee0/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2022-08-26',
 'seats_number': 5,
 'seller_name': None,
 'title': 'OPEL CORSA 1.2T XHL 74kW (100CV) Edition',
 'transmission': 'manual',
 'trim': '1.2T XHL 74kW (100CV) Edition',
 'vin': None,
 'year': 2022}
2025-12-05 20:03:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789100%27 HTTP/1.1" 200 None
2025-12-05 20:03:53 [ayvens.es] INFO: Saving data for 1789100: {'created_time': 1764965033.874408, 'last_price_update_time': 1764965033.874417, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:53 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789100') HTTP/1.1" 204 0
2025-12-05 20:03:53 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: c350f53b-6cd1-5441-acf9-4b9c13e8fb06 with identifier: 1789101
2025-12-05 20:03:53 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: c350f53b-6cd1-5441-acf9-4b9c13e8fb06 with identifier: 1789101
2025-12-05 20:03:54 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'citroën' containing 'citroën' and model 'jumpy' containing 'jumpy'
2025-12-05 20:03:54 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: c350f53b-6cd1-5441-acf9-4b9c13e8fb06 spider: ayvens.es
2025-12-05 20:03:54 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: c350f53b-6cd1-5441-acf9-4b9c13e8fb06 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'citroën' containing 'citroën' and model 'jumpy' containing 'jumpy'
{'auction_title': '',
 'c_o_2_emission_value': 174,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': 'c350f53b-6cd1-5441-acf9-4b9c13e8fb06',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'CITROËN',
 'model': 'JUMPY',
 'odometer': 169050,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_c350f53b-6cd1-5441-acf9-4b9c13e8fb06/c350f53b-6cd1-5441-acf9-4b9c13e8fb06_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 339087,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/880/059bbe3af11d4fe7929af61236492554/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-08-03',
 'seats_number': 2,
 'seller_name': None,
 'title': 'CITROËN JUMPY Talla M BlueHDi 100 S&S 6v Control',
 'transmission': 'manual',
 'trim': 'Talla M BlueHDi 100 S&S 6v Control',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789101%27 HTTP/1.1" 200 None
2025-12-05 20:03:54 [ayvens.es] INFO: Saving data for 1789101: {'created_time': 1764965034.16188, 'last_price_update_time': 1764965034.161892, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789101') HTTP/1.1" 204 0
2025-12-05 20:03:54 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/513/f83615107da9413da4381ec4c20ae5ee/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:54 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://medias-carmarket.ayvens.com/lots/ald/197/8fa52c0613304399a1d776487b17a28f/images/large/1.jpg> (referer: https://api.aldcarmarket.com/graphql/saleevents)
2025-12-05 20:03:54 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:54 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:54 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:54 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0c78d10>
2025-12-05 20:03:54 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:54 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:54 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1971
2025-12-05 20:03:54 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:54 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:54 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:54 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_0ea10ae5-c329-548e-a5c9-4031c5f1e75d/0ea10ae5-c329-548e-a5c9-4031c5f1e75d_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:54 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F375%252Fcc0e4ffc0abd4a739b641957a1fd25b2%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:54 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/375/cc0e4ffc0abd4a739b641957a1fd25b2/images/large/1.jpg saved to cache
2025-12-05 20:03:54 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 0ea10ae5-c329-548e-a5c9-4031c5f1e75d
2025-12-05 20:03:54 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 0ea10ae5-c329-548e-a5c9-4031c5f1e75d, skipping ID generation
2025-12-05 20:03:54 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 0ea10ae5-c329-548e-a5c9-4031c5f1e75d with scrape type 1
2025-12-05 20:03:54 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 0ea10ae5-c329-548e-a5c9-4031c5f1e75d from tracking dictionaries
2025-12-05 20:03:54 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 0ea10ae5-c329-548e-a5c9-4031c5f1e75d sending to next pipeline
2025-12-05 20:03:54 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 0ea10ae5-c329-548e-a5c9-4031c5f1e75d, identifier: 1789102
2025-12-05 20:03:54 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:54 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:54 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:54 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b045b290>
2025-12-05 20:03:54 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:54 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:54 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1965
2025-12-05 20:03:54 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:54 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:54 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:55 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_83944ffd-c45c-555a-a77d-699661e66cb1/83944ffd-c45c-555a-a77d-699661e66cb1_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F771%252Fa98fb86df07440c197d9d4b07192419d%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:55 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/771/a98fb86df07440c197d9d4b07192419d/images/large/1.jpg saved to cache
2025-12-05 20:03:55 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 83944ffd-c45c-555a-a77d-699661e66cb1
2025-12-05 20:03:55 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 83944ffd-c45c-555a-a77d-699661e66cb1, skipping ID generation
2025-12-05 20:03:55 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 83944ffd-c45c-555a-a77d-699661e66cb1 with scrape type 1
2025-12-05 20:03:55 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 83944ffd-c45c-555a-a77d-699661e66cb1 from tracking dictionaries
2025-12-05 20:03:55 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 83944ffd-c45c-555a-a77d-699661e66cb1 sending to next pipeline
2025-12-05 20:03:55 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 83944ffd-c45c-555a-a77d-699661e66cb1, identifier: 1789103
2025-12-05 20:03:55 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 0ea10ae5-c329-548e-a5c9-4031c5f1e75d with identifier: 1789102
2025-12-05 20:03:55 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 0ea10ae5-c329-548e-a5c9-4031c5f1e75d with identifier: 1789102
2025-12-05 20:03:55 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'ford' containing 'ford' and model 'transit courier' containing 'courier'
2025-12-05 20:03:55 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 0ea10ae5-c329-548e-a5c9-4031c5f1e75d spider: ayvens.es
2025-12-05 20:03:55 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 0ea10ae5-c329-548e-a5c9-4031c5f1e75d due to rule: makes_models_not_allowed.json, error: Rejected due to make 'ford' containing 'ford' and model 'transit courier' containing 'courier'
{'auction_title': '',
 'c_o_2_emission_value': 130,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '0ea10ae5-c329-548e-a5c9-4031c5f1e75d',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'FORD',
 'model': 'TRANSIT COURIER',
 'odometer': 93663,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_0ea10ae5-c329-548e-a5c9-4031c5f1e75d/0ea10ae5-c329-548e-a5c9-4031c5f1e75d_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 355142,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/375/cc0e4ffc0abd4a739b641957a1fd25b2/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2021-10-19',
 'seats_number': 2,
 'seller_name': None,
 'title': 'FORD TRANSIT COURIER Van 1.5 TDCi 56kW Trend',
 'transmission': 'manual',
 'trim': 'Van 1.5 TDCi 56kW Trend',
 'vin': None,
 'year': 2021}
2025-12-05 20:03:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789102%27 HTTP/1.1" 200 None
2025-12-05 20:03:55 [ayvens.es] INFO: Saving data for 1789102: {'created_time': 1764965035.459873, 'last_price_update_time': 1764965035.459884, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789102') HTTP/1.1" 204 0
2025-12-05 20:03:55 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 83944ffd-c45c-555a-a77d-699661e66cb1 with identifier: 1789103
2025-12-05 20:03:55 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 83944ffd-c45c-555a-a77d-699661e66cb1 with identifier: 1789103
2025-12-05 20:03:55 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: Rejected due to make 'ford' containing 'ford' and model 'transit courier' containing 'courier'
2025-12-05 20:03:55 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: MAKES_MODELS_NOT_ALLOWED failed for item: 83944ffd-c45c-555a-a77d-699661e66cb1 spider: ayvens.es
2025-12-05 20:03:55 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 83944ffd-c45c-555a-a77d-699661e66cb1 due to rule: makes_models_not_allowed.json, error: Rejected due to make 'ford' containing 'ford' and model 'transit courier' containing 'courier'
{'auction_title': '',
 'c_o_2_emission_value': 130,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 4,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '83944ffd-c45c-555a-a77d-699661e66cb1',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'FORD',
 'model': 'TRANSIT COURIER',
 'odometer': 120879,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_83944ffd-c45c-555a-a77d-699661e66cb1/83944ffd-c45c-555a-a77d-699661e66cb1_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 346650,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/771/a98fb86df07440c197d9d4b07192419d/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2020-11-23',
 'seats_number': 2,
 'seller_name': None,
 'title': 'FORD TRANSIT COURIER Van 1.5 TDCi 56kW Trend',
 'transmission': 'manual',
 'trim': 'Van 1.5 TDCi 56kW Trend',
 'vin': None,
 'year': 2020}
2025-12-05 20:03:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789103%27 HTTP/1.1" 200 None
2025-12-05 20:03:55 [ayvens.es] INFO: Saving data for 1789103: {'created_time': 1764965035.718148, 'last_price_update_time': 1764965035.71816, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:55 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789103') HTTP/1.1" 204 0
2025-12-05 20:03:55 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:55 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:55 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:55 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a411d0>
2025-12-05 20:03:55 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:55 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:55 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1969
2025-12-05 20:03:55 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:56 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:56 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:56 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_42a795cb-5341-5e0e-bfae-529bbbbe2830/42a795cb-5341-5e0e-bfae-529bbbbe2830_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:56 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:56 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F513%252Ff83615107da9413da4381ec4c20ae5ee%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:56 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/513/f83615107da9413da4381ec4c20ae5ee/images/large/1.jpg saved to cache
2025-12-05 20:03:56 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 42a795cb-5341-5e0e-bfae-529bbbbe2830
2025-12-05 20:03:56 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 42a795cb-5341-5e0e-bfae-529bbbbe2830, skipping ID generation
2025-12-05 20:03:56 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 42a795cb-5341-5e0e-bfae-529bbbbe2830 with scrape type 1
2025-12-05 20:03:56 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 42a795cb-5341-5e0e-bfae-529bbbbe2830 from tracking dictionaries
2025-12-05 20:03:56 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 42a795cb-5341-5e0e-bfae-529bbbbe2830 sending to next pipeline
2025-12-05 20:03:56 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 42a795cb-5341-5e0e-bfae-529bbbbe2830, identifier: 1789104
2025-12-05 20:03:56 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2025-12-05 20:03:56 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2025-12-05 20:03:56 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2025-12-05 20:03:56 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7f34b0a42ad0>
2025-12-05 20:03:56 [common.azure_blobstore] INFO: Creating container if not exists
2025-12-05 20:03:56 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2025-12-05 20:03:56 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 1970
2025-12-05 20:03:56 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2025-12-05 20:03:56 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2025-12-05 20:03:56 [common.azure_blobstore] INFO: Container already exists, skipping creation
2025-12-05 20:03:56 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_92f550bf-afa8-554e-a076-5488bc8e638b/92f550bf-afa8-554e-a076-5488bc8e638b_image_0.jpg HTTP/1.1" 201 0
2025-12-05 20:03:56 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:56 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='ayvens.es',RowKey='https%3A%252F%252Fmedias-carmarket.ayvens.com%252Flots%252Fald%252F197%252F8fa52c0613304399a1d776487b17a28f%252Fimages%252Flarge%252F1.jpg') HTTP/1.1" 204 0
2025-12-05 20:03:56 [crawlers.middlewares.photo_download_middleware] INFO: Image https://medias-carmarket.ayvens.com/lots/ald/197/8fa52c0613304399a1d776487b17a28f/images/large/1.jpg saved to cache
2025-12-05 20:03:56 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 92f550bf-afa8-554e-a076-5488bc8e638b
2025-12-05 20:03:56 [crawlers.middlewares.id_gen_middleware] DEBUG: Item already has an ID: 92f550bf-afa8-554e-a076-5488bc8e638b, skipping ID generation
2025-12-05 20:03:56 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 92f550bf-afa8-554e-a076-5488bc8e638b with scrape type 1
2025-12-05 20:03:56 [crawlers.middlewares.photo_download_middleware] INFO: Removing item 92f550bf-afa8-554e-a076-5488bc8e638b from tracking dictionaries
2025-12-05 20:03:56 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 92f550bf-afa8-554e-a076-5488bc8e638b sending to next pipeline
2025-12-05 20:03:56 [crawlers.middlewares.monitoring_spider_middleware] INFO: Spider: ayvens.es, Processed item id: 92f550bf-afa8-554e-a076-5488bc8e638b, identifier: 1789105
2025-12-05 20:03:56 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 42a795cb-5341-5e0e-bfae-529bbbbe2830 with identifier: 1789104
2025-12-05 20:03:56 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 42a795cb-5341-5e0e-bfae-529bbbbe2830 with identifier: 1789104
2025-12-05 20:03:56 [crawlers.pipelines.item_rules_pipeline] INFO: Validation error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'nissan', 'model': 'nv200', 'trim': '1.5dci 66kw (90cv) comfort mam', 'title': 'nissan nv200 1.5dci 66kw (90cv) comfort mam', 'price': 0, 'odometer': 367522, 'odometer_reading_unit': 'km', 'registration_date': '2019-08-27', 'year': 2019, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 2, 'vin': None, 'license_plate': None, 'engine': '1.5l', 'c_o_2_emission_value': 131, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '42a795cb-5341-5e0e-bfae-529bbbbe2830', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_42a795cb-5341-5e0e-bfae-529bbbbe2830/42a795cb-5341-5e0e-bfae-529bbbbe2830_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/513/f83615107da9413da4381ec4c20ae5ee/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 333781}], 'current_year_diff': 6, 'age_in_months': 75, 'make_model': 'nissan nv200', 'sanitized_make': 'nissan', 'odometer_in_km': 367522}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 75,
     'auction_title': '',
     'c_o_2_emission_value': 131,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 6,
     'doors_number': 5,
     'engine': '1.5l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': '42a795cb-5341-5e0e-bfae-529bbbbe2830',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
     'listing': 'auction',
     'location': None,
     'make': 'nissan',
     'make_model': 'nissan nv200',
     'model': 'nv200',
     'odometer': 367522,
     'odometer_in_km': 367522,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_42a795cb-5341-5e0e-bfae-529bbbbe2830/42a795cb-5341-5e0e-bfae-529bbbbe2830_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 333781,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/513/f83615107da9413da4381ec4c20ae5ee/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2019-08-27',
     'sanitized_make': 'nissan',
     'seats_number': 2,
     'seller_name': None,
     'title': 'nissan nv200 1.5dci 66kw (90cv) comfort mam',
     'transmission': 'manual',
     'trim': '1.5dci 66kw (90cv) comfort mam',
     'vin': None,
     'year': 2019}
2025-12-05 20:03:56 [crawlers.pipelines.item_rules_pipeline] INFO: rule_title: HIGH_MILEAGE failed for item: 42a795cb-5341-5e0e-bfae-529bbbbe2830 spider: ayvens.es
2025-12-05 20:03:56 [scrapy.core.scraper] INFO: Dropped: Spider: ayvens.es, Dropping listing: 42a795cb-5341-5e0e-bfae-529bbbbe2830 due to rule: high_mileage.json, error: False schema does not allow {'auction_title': '', 'is_operable': True, 'is_damaged': False, 'color': 'blanco', 'original_photo_count': 1, 'seller_name': None, 'price_includes_vat': False, 'make': 'nissan', 'model': 'nv200', 'trim': '1.5dci 66kw (90cv) comfort mam', 'title': 'nissan nv200 1.5dci 66kw (90cv) comfort mam', 'price': 0, 'odometer': 367522, 'odometer_reading_unit': 'km', 'registration_date': '2019-08-27', 'year': 2019, 'fuel_type': 'diesel', 'listing': 'auction', 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766', 'expiration_date': '2025-12-09t12:00:00z', 'is_vat_included': True, 'is_vat_deductible': False, 'doors_number': 5, 'seats_number': 2, 'vin': None, 'license_plate': None, 'engine': '1.5l', 'c_o_2_emission_value': 131, 'origin_country_code': 'es', 'transmission': 'manual', 'location': None, 'currency': 'eur', 'id': '42a795cb-5341-5e0e-bfae-529bbbbe2830', 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_42a795cb-5341-5e0e-bfae-529bbbbe2830/42a795cb-5341-5e0e-bfae-529bbbbe2830_image_0.jpg', 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/513/f83615107da9413da4381ec4c20ae5ee/images/large/1.jpg', 'Width': 1024, 'Height': 768, 'Extension': 'jpg', 'FileSizeInBytes': 333781}], 'current_year_diff': 6, 'age_in_months': 75, 'make_model': 'nissan nv200', 'sanitized_make': 'nissan', 'odometer_in_km': 367522}

Failed validating None in schema:
    False

On instance:
    {'age_in_months': 75,
     'auction_title': '',
     'c_o_2_emission_value': 131,
     'color': 'blanco',
     'currency': 'eur',
     'current_year_diff': 6,
     'doors_number': 5,
     'engine': '1.5l',
     'expiration_date': '2025-12-09t12:00:00z',
     'fuel_type': 'diesel',
     'id': '42a795cb-5341-5e0e-bfae-529bbbbe2830',
     'is_damaged': False,
     'is_operable': True,
     'is_vat_deductible': False,
     'is_vat_included': True,
     'license_plate': None,
     'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
     'listing': 'auction',
     'location': None,
     'make': 'nissan',
     'make_model': 'nissan nv200',
     'model': 'nv200',
     'odometer': 367522,
     'odometer_in_km': 367522,
     'odometer_reading_unit': 'km',
     'origin_country_code': 'es',
     'original_photo_count': 1,
     'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_42a795cb-5341-5e0e-bfae-529bbbbe2830/42a795cb-5341-5e0e-bfae-529bbbbe2830_image_0.jpg',
                 'Extension': 'jpg',
                 'FileSizeInBytes': 333781,
                 'Height': 768,
                 'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/513/f83615107da9413da4381ec4c20ae5ee/images/large/1.jpg',
                 'Width': 1024}],
     'price': 0,
     'price_includes_vat': False,
     'registration_date': '2019-08-27',
     'sanitized_make': 'nissan',
     'seats_number': 2,
     'seller_name': None,
     'title': 'nissan nv200 1.5dci 66kw (90cv) comfort mam',
     'transmission': 'manual',
     'trim': '1.5dci 66kw (90cv) comfort mam',
     'vin': None,
     'year': 2019}
{'auction_title': '',
 'c_o_2_emission_value': 131,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '42a795cb-5341-5e0e-bfae-529bbbbe2830',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'NISSAN',
 'model': 'NV200',
 'odometer': 367522,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_42a795cb-5341-5e0e-bfae-529bbbbe2830/42a795cb-5341-5e0e-bfae-529bbbbe2830_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 333781,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/513/f83615107da9413da4381ec4c20ae5ee/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2019-08-27',
 'seats_number': 2,
 'seller_name': None,
 'title': 'NISSAN NV200 1.5dCi 66kW (90CV) COMFORT Mam',
 'transmission': 'manual',
 'trim': '1.5dCi 66kW (90CV) COMFORT Mam',
 'vin': None,
 'year': 2019}
2025-12-05 20:03:56 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:56 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789104%27 HTTP/1.1" 200 None
2025-12-05 20:03:56 [ayvens.es] INFO: Saving data for 1789104: {'created_time': 1764965036.940605, 'last_price_update_time': 1764965036.940614, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:56 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:56 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789104') HTTP/1.1" 204 0
2025-12-05 20:03:56 [crawlers.pipelines.translation_pipeline] INFO: Spider: ayvens.es, Translating item: 92f550bf-afa8-554e-a076-5488bc8e638b with identifier: 1789105
2025-12-05 20:03:56 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: ayvens.es, Applying rules to item: 92f550bf-afa8-554e-a076-5488bc8e638b with identifier: 1789105
2025-12-05 20:03:57 [crawlers.pipelines.post_to_api] INFO: Spider: ayvens.es, Posting item: 92f550bf-afa8-554e-a076-5488bc8e638b with identifier: 1789105 to the API
2025-12-05 20:03:57 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2025-12-05 20:03:57 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2025-12-05 20:03:57 [scrapy.core.scraper] DEBUG: Scraped from <200 https://medias-carmarket.ayvens.com/lots/ald/197/8fa52c0613304399a1d776487b17a28f/images/large/1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 99,
 'color': 'Blanco',
 'currency': 'EUR',
 'doors_number': 5,
 'engine': '1.5L',
 'expiration_date': '2025-12-09T12:00:00Z',
 'fuel_type': 'diesel',
 'id': '92f550bf-afa8-554e-a076-5488bc8e638b',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'is_vat_included': True,
 'license_plate': None,
 'link_to_web_offer': 'https://carmarket.ayvens.com/en-gb/sales/37766',
 'listing': 'auction',
 'location': None,
 'make': 'NISSAN',
 'model': 'QASHQAI',
 'odometer': 131328,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'ES',
 'original_photo_count': 1,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_92f550bf-afa8-554e-a076-5488bc8e638b/92f550bf-afa8-554e-a076-5488bc8e638b_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 337956,
             'Height': 768,
             'OriginUrl': 'https://medias-carmarket.ayvens.com/lots/ald/197/8fa52c0613304399a1d776487b17a28f/images/large/1.jpg',
             'Width': 1024}],
 'price': 0,
 'price_includes_vat': False,
 'registration_date': '2018-05-31',
 'seats_number': 5,
 'seller_name': None,
 'title': 'NISSAN QASHQAI dCi 81 kW (110 CV) N-CONNECTA',
 'transmission': 'manual',
 'trim': 'dCi 81 kW (110 CV) N-CONNECTA',
 'vin': None,
 'year': 2018}
2025-12-05 20:03:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20RowKey%20eq%20%271789105%27 HTTP/1.1" 200 None
2025-12-05 20:03:57 [ayvens.es] INFO: Saving data for 1789105: {'created_time': 1764965037.360803, 'last_price_update_time': 1764965037.360815, 'auction_closing_time': 1765281600.0}
2025-12-05 20:03:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:57 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='ayvens.es',RowKey='1789105') HTTP/1.1" 204 0
2025-12-05 20:03:58 [scrapy.core.engine] DEBUG: Crawled (200) <POST https://api.aldcarmarket.com/graphql/saleevents> (referer: https://carmarket.ayvens.com/) ['zyte-api']
2025-12-05 20:03:58 [scrapy.core.engine] INFO: Closing spider (finished)
2025-12-05 20:03:58 [ayvens.es] INFO: ayvens.es Crawl ended with reason finished, scrape types: {<ScrapeType.NEW: 1>: 2, <ScrapeType.NEW_DUPLICATE_ID: 4>: 0, <ScrapeType.PRICE_UPDATE: 2>: 0, <ScrapeType.AUCTION_UPDATE: 3>: 0, <ScrapeType.SKIPPED: 0>: 0}
2025-12-05 20:03:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2025-12-05 20:03:58 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27ayvens.es%27%20and%20last_price_update_time%20lt%201764533038 HTTP/1.1" 200 None
2025-12-05 20:03:59 [scrapy.extensions.feedexport] INFO: Stored jsonlines feed (134 items) in: file:///var/lib/scrapyd/items/sourcing_v2/ayvens.es/task_54_2025-12-05T20_00_00.jl
2025-12-05 20:03:59 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 110586,
 'downloader/request_count': 230,
 'downloader/request_method_count/GET': 226,
 'downloader/request_method_count/POST': 4,
 'downloader/response_bytes': 81925606,
 'downloader/response_count': 230,
 'downloader/response_status_count/200': 230,
 'elapsed_time_seconds': 231.19345,
 'feedexport/success_count/FileFeedStorage': 1,
 'finish_reason': 'finished',
 'finish_time': datetime.datetime(2025, 12, 5, 20, 3, 58, 910405, tzinfo=datetime.timezone.utc),
 'item_dropped_count': 92,
 'item_dropped_reasons_count/DropItem': 92,
 'item_scraped_count': 134,
 'log_count/DEBUG': 4551,
 'log_count/INFO': 5471,
 'memusage/max': 183316480,
 'memusage/startup': 126992384,
 'photo_download_count': 225,
 'playwright/context_count': 1,
 'playwright/context_count/max_concurrent': 1,
 'playwright/context_count/persistent/False': 1,
 'playwright/context_count/remote/False': 1,
 'playwright/page_count': 1,
 'playwright/page_count/max_concurrent': 1,
 'playwright/request_count': 130,
 'playwright/request_count/aborted': 13,
 'playwright/request_count/method/GET': 126,
 'playwright/request_count/method/POST': 4,
 'playwright/request_count/navigation': 2,
 'playwright/request_count/resource_type/document': 2,
 'playwright/request_count/resource_type/fetch': 1,
 'playwright/request_count/resource_type/font': 16,
 'playwright/request_count/resource_type/image': 25,
 'playwright/request_count/resource_type/script': 80,
 'playwright/request_count/resource_type/stylesheet': 1,
 'playwright/request_count/resource_type/xhr': 5,
 'playwright/response_count': 114,
 'playwright/response_count/method/GET': 111,
 'playwright/response_count/method/POST': 3,
 'playwright/response_count/resource_type/document': 2,
 'playwright/response_count/resource_type/fetch': 1,
 'playwright/response_count/resource_type/font': 16,
 'playwright/response_count/resource_type/image': 10,
 'playwright/response_count/resource_type/script': 80,
 'playwright/response_count/resource_type/stylesheet': 1,
 'playwright/response_count/resource_type/xhr': 4,
 'request_depth_max': 4,
 'response_received_count': 5,
 'scheduler/dequeued': 230,
 'scheduler/dequeued/memory': 230,
 'scheduler/enqueued': 230,
 'scheduler/enqueued/memory': 230,
 'scrapy-zyte-api/429': 0,
 'scrapy-zyte-api/attempts': 4,
 'scrapy-zyte-api/error_ratio': 0.0,
 'scrapy-zyte-api/errors': 0,
 'scrapy-zyte-api/fatal_errors': 0,
 'scrapy-zyte-api/mean_connection_seconds': 4.917576481006108,
 'scrapy-zyte-api/mean_response_seconds': 4.923854045249755,
 'scrapy-zyte-api/processed': 4,
 'scrapy-zyte-api/request_args/customHttpRequestHeaders': 4,
 'scrapy-zyte-api/request_args/experimental.responseCookies': 4,
 'scrapy-zyte-api/request_args/httpRequestBody': 4,
 'scrapy-zyte-api/request_args/httpRequestMethod': 4,
 'scrapy-zyte-api/request_args/httpResponseBody': 4,
 'scrapy-zyte-api/request_args/httpResponseHeaders': 4,
 'scrapy-zyte-api/request_args/url': 4,
 'scrapy-zyte-api/status_codes/200': 4,
 'scrapy-zyte-api/success': 4,
 'scrapy-zyte-api/success_ratio': 1.0,
 'scrapy-zyte-api/throttle_ratio': 0.0,
 'start_time': datetime.datetime(2025, 12, 5, 20, 0, 7, 716955, tzinfo=datetime.timezone.utc)}
2025-12-05 20:03:59 [scrapy.core.engine] INFO: Spider closed (finished)