2026-03-25 17:18:04 [scrapy.utils.log] INFO: Scrapy 2.11.2 started (bot: SourcingV2)
2026-03-25 17:18:04 [scrapy.utils.log] INFO: Versions: lxml 5.2.2.0, libxml2 2.12.6, cssselect 1.2.0, parsel 1.9.1, w3lib 2.1.2, Twisted 24.3.0, Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0], pyOpenSSL 24.1.0 (OpenSSL 3.2.1 30 Jan 2024), cryptography 42.0.7, Platform Linux-5.15.0-1098-azure-x86_64-with-glibc2.36
2026-03-25 17:18:04 [auto_selling_autotrader.co.uk] INFO: Starting auto_selling_autotrader.co.uk spider
2026-03-25 17:18:04 [auto_selling_autotrader.co.uk] INFO: Quick mode enabled: batching detail requests in groups of 10
2026-03-25 17:18:04 [auto_selling_autotrader.co.uk] INFO: Limiting photo downloads to 1 per item
2026-03-25 17:18:04 [auto_selling_autotrader.co.uk] INFO: _job: 94508962286e11f18f76c264925fbcfa, SCRAPY_JOB: None
2026-03-25 17:18:04 [scrapy.addons] INFO: Enabled addons:
[]
2026-03-25 17:18:04 [asyncio] DEBUG: Using selector: EpollSelector
2026-03-25 17:18:04 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.asyncioreactor.AsyncioSelectorReactor
2026-03-25 17:18:04 [scrapy.utils.log] DEBUG: Using asyncio event loop: asyncio.unix_events._UnixSelectorEventLoop
2026-03-25 17:18:04 [scrapy.extensions.telnet] INFO: Telnet Password: 2ff68253712e01b5
2026-03-25 17:18:04 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
'scrapy.extensions.telnet.TelnetConsole',
'scrapy.extensions.memusage.MemoryUsage',
'scrapy.extensions.feedexport.FeedExporter',
'scrapy.extensions.logstats.LogStats',
'scrapy.extensions.closespider.CloseSpider']
2026-03-25 17:18:04 [scrapy.crawler] INFO: Overridden settings:
{'BOT_NAME': 'SourcingV2',
'CLOSESPIDER_TIMEOUT': 3600,
'DOWNLOAD_MAXSIZE': 52428800,
'DOWNLOAD_WARNSIZE': 10485760,
'FEED_EXPORT_ENCODING': 'utf-8',
'LOG_FILE': '/var/log/scrapyd/logs/sourcing_v2/auto_selling_autotrader.co.uk/94508962286e11f18f76c264925fbcfa.log',
'LOG_FORMATTER': 'crawlers.log_formatter.SourcingLogFormatter',
'MEMUSAGE_LIMIT_MB': 2048,
'MEMUSAGE_WARNING_MB': 1536,
'NEWSPIDER_MODULE': 'spiders',
'REQUEST_FINGERPRINTER_CLASS': 'scrapy_zyte_api.ScrapyZyteAPIRequestFingerprinter',
'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7',
'SPIDER_MODULES': ['spiders', 'auth_check'],
'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor',
'USER_AGENT': ''}
2026-03-25 17:18:04 [scrapy_zyte_api.handler] INFO: Using a Zyte API key starting with '5857011'
2026-03-25 17:18:04 [scrapy_zyte_api.handler] INFO: Using a Zyte API key starting with '5857011'
2026-03-25 17:18:04 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.offsite.OffsiteMiddleware',
'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
'scrapy_zyte_api.ScrapyZyteAPIDownloaderMiddleware',
'scrapy.downloadermiddlewares.retry.RetryMiddleware',
'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
'scrapy.downloadermiddlewares.stats.DownloaderStats']
2026-03-25 17:18:04 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Setting up IdGenerationMiddleware for auto_selling
2026-03-25 17:18:04 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
'scrapy_zyte_api.ScrapyZyteAPISpiderMiddleware',
'scrapy.spidermiddlewares.referer.RefererMiddleware',
'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
'scrapy.spidermiddlewares.depth.DepthMiddleware',
'crawlers.middlewares.photo_download_middleware.PhotoDownloadMiddleware',
'crawlers.auto_selling.middlewares.id_gen_middleware.IdGenMiddleware']
2026-03-25 17:18:04 [scrapy.middleware] INFO: Enabled item pipelines:
['crawlers.pipelines.post_to_azurebus_auto_selling.AzureBusPipelineAutoSelling',
'crawlers.pipelines.save_to_cdlq_db_auto_selling.SaveToCdlqDbAutoSelling']
2026-03-25 17:18:04 [scrapy.core.engine] INFO: Spider opened
2026-03-25 17:18:04 [azure.identity._credentials.environment] INFO: Incomplete environment configuration for EnvironmentCredential. These variables are set: AZURE_TENANT_ID, AZURE_CLIENT_ID
2026-03-25 17:18:04 [common.service_bus.service_bus_client_factory] INFO: ServiceBusClient initialized via DefaultAzureCredential for alx-production-bus.servicebus.windows.net
2026-03-25 17:18:04 [crawlers.pipelines.base_azure_bus_pipeline] INFO: [AzureBusPipeline] Connected to topic auto.sourcing-v2-quickmode
2026-03-25 17:18:04 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-03-25 17:18:04 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2026-03-25 17:18:04 [scrapy.extensions.memusage] INFO: Peak memory usage is 140MiB
2026-03-25 17:18:04 [scrapy-playwright] INFO: Starting download handler
2026-03-25 17:18:04 [scrapy-playwright] INFO: Starting download handler
2026-03-25 17:18:09 [auto_selling_autotrader.co.uk] INFO: Scraper started with retailer ID: 10010454 and multipost_profile_id: d3d901f6-9a58-42b4-993f-0420bd220238 and job_id: 94508962286e11f18f76c264925fbcfa and url: https://www.autotrader.co.uk/dealers/insta-cars-10010454
2026-03-25 17:18:09 [auto_selling_autotrader.co.uk] INFO: Getting list for retailer ID: 10010454, page: 1
2026-03-25 17:18:09 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 1st time calling it.
2026-03-25 17:18:10 [scrapy.core.engine] DEBUG: Crawled (200) <POST https://www.autotrader.co.uk/at-gateway?opname=SearchResultsListingsGridQuery&opname=SearchResultsFacetsWithGroupsQuery> (referer: None) ['zyte-api']
2026-03-25 17:18:11 [auto_selling_autotrader.co.uk] DEBUG: response: <200 https://www.autotrader.co.uk/at-gateway?opname=SearchResultsListingsGridQuery&opname=SearchResultsFacetsWithGroupsQuery>
2026-03-25 17:18:11 [auto_selling_autotrader.co.uk] INFO: Page: 1/1, total results: 16
2026-03-25 17:18:11 [azure.identity._credentials.environment] INFO: Incomplete environment configuration for EnvironmentCredential. These variables are set: AZURE_TENANT_ID, AZURE_CLIENT_ID
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 2111
2026-03-25 17:18:11 [azure.identity._credentials.chained] INFO: DefaultAzureCredential acquired a token from WorkloadIdentityCredential
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxsourcingstorageprod.table.core.windows.net:443
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603190849828%27 HTTP/1.1" 200 None
2026-03-25 17:18:11 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202603190849828: 1
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603190849335%27 HTTP/1.1" 200 None
2026-03-25 17:18:11 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202603190849335: 1
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603100581104%27 HTTP/1.1" 200 None
2026-03-25 17:18:11 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202603100581104: 1
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603100580971%27 HTTP/1.1" 200 None
2026-03-25 17:18:11 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202603100580971: 1
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603100580851%27 HTTP/1.1" 200 None
2026-03-25 17:18:11 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202603100580851: 1
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603070485608%27 HTTP/1.1" 200 None
2026-03-25 17:18:11 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202603070485608: 1
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603050441201%27 HTTP/1.1" 200 None
2026-03-25 17:18:11 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202603050441201: 1
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603030380804%27 HTTP/1.1" 200 None
2026-03-25 17:18:11 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202603030380804: 1
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602270283424%27 HTTP/1.1" 200 None
2026-03-25 17:18:11 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202602270283424: 1
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602190061263%27 HTTP/1.1" 200 None
2026-03-25 17:18:11 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202602190061263: 1
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602179992097%27 HTTP/1.1" 200 None
2026-03-25 17:18:11 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202602179992097: 1
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602139919458%27 HTTP/1.1" 200 None
2026-03-25 17:18:11 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202602139919458: 1
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602129879480%27 HTTP/1.1" 200 None
2026-03-25 17:18:11 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202602129879480: 1
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202601319572526%27 HTTP/1.1" 200 None
2026-03-25 17:18:11 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202601319572526: 1
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202512228751456%27 HTTP/1.1" 200 None
2026-03-25 17:18:11 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202512228751456: 1
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:11 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202510076951486%27 HTTP/1.1" 200 None
2026-03-25 17:18:11 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202510076951486: 1
2026-03-25 17:18:11 [auto_selling_autotrader.co.uk] INFO: Quick mode: batching 10 detail requests (batch 1)
2026-03-25 17:18:11 [auto_selling_autotrader.co.uk] INFO: Quick mode: batching 6 detail requests (batch 2)
2026-03-25 17:18:11 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 1st time calling it.
2026-03-25 17:18:11 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 1st time calling it.
2026-03-25 17:18:15 [scrapy.core.engine] DEBUG: Crawled (200) <POST https://www.autotrader.co.uk/at-graphql> (referer: https://www.autotrader.co.uk/at-gateway?opname=SearchResultsListingsGridQuery&opname=SearchResultsFacetsWithGroupsQuery) ['zyte-api']
2026-03-25 17:18:15 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-25 17:18:15 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: d9dc8bf4-72ae-5ef0-80f7-f6c9e7e7c267 with identifier: 202603190849828
2026-03-25 17:18:15 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item d9dc8bf4-72ae-5ef0-80f7-f6c9e7e7c267
2026-03-25 17:18:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F54f697b60a284893b9de4aecb8103d9e.jpg%27 HTTP/1.1" 200 None
2026-03-25 17:18:15 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/54f697b60a284893b9de4aecb8103d9e.jpg not downloaded yet
2026-03-25 17:18:15 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-25 17:18:15 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: 381a9612-b895-5616-bdec-cb08a2179d8e with identifier: 202603190849335
2026-03-25 17:18:15 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 381a9612-b895-5616-bdec-cb08a2179d8e
2026-03-25 17:18:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F07116b4e0bc240948552cb9891d14b7d.jpg%27 HTTP/1.1" 200 None
2026-03-25 17:18:15 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/07116b4e0bc240948552cb9891d14b7d.jpg not downloaded yet
2026-03-25 17:18:15 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-25 17:18:15 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: 0a0072cb-85ca-5df5-b1ae-ff4248114fb0 with identifier: 202603100581104
2026-03-25 17:18:15 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 0a0072cb-85ca-5df5-b1ae-ff4248114fb0
2026-03-25 17:18:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F6f58fb4f11c044148c5921b16b75950b.jpg%27 HTTP/1.1" 200 None
2026-03-25 17:18:15 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/6f58fb4f11c044148c5921b16b75950b.jpg not downloaded yet
2026-03-25 17:18:15 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-25 17:18:15 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: 9810d4ec-5e18-5c77-be21-0e0b402bbf10 with identifier: 202603100580971
2026-03-25 17:18:15 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 9810d4ec-5e18-5c77-be21-0e0b402bbf10
2026-03-25 17:18:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252Ffaf9fb3a50b44399a16b45e5d0f3641c.jpg%27 HTTP/1.1" 200 None
2026-03-25 17:18:15 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/faf9fb3a50b44399a16b45e5d0f3641c.jpg not downloaded yet
2026-03-25 17:18:15 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-25 17:18:15 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: 319b631e-b8f2-5231-b252-77d7b4300540 with identifier: 202603100580851
2026-03-25 17:18:15 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 319b631e-b8f2-5231-b252-77d7b4300540
2026-03-25 17:18:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252Fe6777e840fa94191ae267cfcf6aae447.jpg%27 HTTP/1.1" 200 None
2026-03-25 17:18:15 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/e6777e840fa94191ae267cfcf6aae447.jpg not downloaded yet
2026-03-25 17:18:15 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-25 17:18:15 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: aa1919c3-9e71-5e72-acfe-19f3a9a82079 with identifier: 202603070485608
2026-03-25 17:18:15 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item aa1919c3-9e71-5e72-acfe-19f3a9a82079
2026-03-25 17:18:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252Fb1926085dd0c4c22bd7ab6e21b84ad4a.jpg%27 HTTP/1.1" 200 None
2026-03-25 17:18:15 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/b1926085dd0c4c22bd7ab6e21b84ad4a.jpg not downloaded yet
2026-03-25 17:18:15 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-25 17:18:15 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: 717a4521-3f4e-5ef1-abb2-baf793b361d5 with identifier: 202603050441201
2026-03-25 17:18:15 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 717a4521-3f4e-5ef1-abb2-baf793b361d5
2026-03-25 17:18:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F207d62219ff941b49596bfd214608968.jpg%27 HTTP/1.1" 200 None
2026-03-25 17:18:15 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/207d62219ff941b49596bfd214608968.jpg not downloaded yet
2026-03-25 17:18:15 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-25 17:18:15 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: 396b41db-eeeb-5ed2-bb8a-d6926295bd20 with identifier: 202603030380804
2026-03-25 17:18:15 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 396b41db-eeeb-5ed2-bb8a-d6926295bd20
2026-03-25 17:18:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F276275b799ce4049b13d77d02f185700.jpg%27 HTTP/1.1" 200 None
2026-03-25 17:18:15 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/276275b799ce4049b13d77d02f185700.jpg not downloaded yet
2026-03-25 17:18:15 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-25 17:18:15 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: adbc7c52-da1d-52ca-ba3c-a136f7894af9 with identifier: 202602270283424
2026-03-25 17:18:15 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item adbc7c52-da1d-52ca-ba3c-a136f7894af9
2026-03-25 17:18:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F8f9352a73cac43d6bfaf45bbcc37f7cd.jpg%27 HTTP/1.1" 200 None
2026-03-25 17:18:15 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/8f9352a73cac43d6bfaf45bbcc37f7cd.jpg not downloaded yet
2026-03-25 17:18:15 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-25 17:18:15 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: dddc58a7-570d-5c91-8f47-bb18b640bb46 with identifier: 202602190061263
2026-03-25 17:18:15 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item dddc58a7-570d-5c91-8f47-bb18b640bb46
2026-03-25 17:18:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252Fd2fe61618b0b489eb597660898d37fb1.jpg%27 HTTP/1.1" 200 None
2026-03-25 17:18:15 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/d2fe61618b0b489eb597660898d37fb1.jpg not downloaded yet
2026-03-25 17:18:15 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/faf9fb3a50b44399a16b45e5d0f3641c.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-25 17:18:15 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/54f697b60a284893b9de4aecb8103d9e.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-25 17:18:15 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/e6777e840fa94191ae267cfcf6aae447.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-25 17:18:15 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/07116b4e0bc240948552cb9891d14b7d.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-25 17:18:15 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/276275b799ce4049b13d77d02f185700.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-25 17:18:15 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/207d62219ff941b49596bfd214608968.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-25 17:18:15 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/8f9352a73cac43d6bfaf45bbcc37f7cd.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-25 17:18:15 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/6f58fb4f11c044148c5921b16b75950b.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-25 17:18:15 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2026-03-25 17:18:15 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2026-03-25 17:18:15 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-25 17:18:15 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fcc2798b590>
2026-03-25 17:18:15 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-25 17:18:15 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2026-03-25 17:18:15 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 2119
2026-03-25 17:18:15 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2026-03-25 17:18:16 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-25 17:18:16 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-25 17:18:16 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_9810d4ec-5e18-5c77-be21-0e0b402bbf10/9810d4ec-5e18-5c77-be21-0e0b402bbf10_image_0.jpg HTTP/1.1" 201 0
2026-03-25 17:18:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252Ffaf9fb3a50b44399a16b45e5d0f3641c.jpg') HTTP/1.1" 204 0
2026-03-25 17:18:16 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/faf9fb3a50b44399a16b45e5d0f3641c.jpg saved to cache
2026-03-25 17:18:16 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 9810d4ec-5e18-5c77-be21-0e0b402bbf10
2026-03-25 17:18:16 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 9810d4ec-5e18-5c77-be21-0e0b402bbf10 sending to next pipeline
2026-03-25 17:18:16 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: 9810d4ec-5e18-5c77-be21-0e0b402bbf10 with identifier 202603100580971
2026-03-25 17:18:16 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: None -> <ConnectionState.START: 0>
2026-03-25 17:18:16 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.START: 0> -> <ConnectionState.HDR_SENT: 2>
2026-03-25 17:18:16 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.HDR_SENT: 2> -> <ConnectionState.HDR_SENT: 2>
2026-03-25 17:18:16 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.HDR_SENT: 2> -> <ConnectionState.OPEN_PIPE: 4>
2026-03-25 17:18:16 [azure.servicebus._pyamqp.session] INFO: Session state changed: <SessionState.UNMAPPED: 0> -> <SessionState.BEGIN_SENT: 1>
2026-03-25 17:18:16 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACHED: 0> -> <LinkState.ATTACH_SENT: 1>
2026-03-25 17:18:16 [azure.servicebus._pyamqp.management_link] INFO: Management link receiver state changed: <LinkState.DETACHED: 0> -> <LinkState.ATTACH_SENT: 1>
2026-03-25 17:18:16 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACHED: 0> -> <LinkState.ATTACH_SENT: 1>
2026-03-25 17:18:16 [azure.servicebus._pyamqp.management_link] INFO: Management link sender state changed: <LinkState.DETACHED: 0> -> <LinkState.ATTACH_SENT: 1>
2026-03-25 17:18:16 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.OPEN_PIPE: 4> -> <ConnectionState.OPEN_SENT: 7>
2026-03-25 17:18:16 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.OPEN_SENT: 7> -> <ConnectionState.OPENED: 9>
2026-03-25 17:18:16 [azure.servicebus._pyamqp.session] INFO: Session state changed: <SessionState.BEGIN_SENT: 1> -> <SessionState.MAPPED: 3>
2026-03-25 17:18:16 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACH_SENT: 1> -> <LinkState.ATTACHED: 3>
2026-03-25 17:18:16 [azure.servicebus._pyamqp.management_link] INFO: Management link receiver state changed: <LinkState.ATTACH_SENT: 1> -> <LinkState.ATTACHED: 3>
2026-03-25 17:18:16 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACH_SENT: 1> -> <LinkState.ATTACHED: 3>
2026-03-25 17:18:16 [azure.servicebus._pyamqp.management_link] INFO: Management link sender state changed: <LinkState.ATTACH_SENT: 1> -> <LinkState.ATTACHED: 3>
2026-03-25 17:18:16 [azure.servicebus._pyamqp.cbs] DEBUG: CBS completed opening with status: <ManagementOpenResult.OK: 1>
2026-03-25 17:18:16 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2026-03-25 17:18:16 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 2017
2026-03-25 17:18:16 [azure.identity._credentials.chained] INFO: DefaultAzureCredential acquired a token from WorkloadIdentityCredential
2026-03-25 17:18:16 [azure.servicebus._pyamqp.cbs] DEBUG: CBS update in progress. Token put time: 1774459096
2026-03-25 17:18:16 [azure.servicebus._pyamqp.cbs] DEBUG: CBS update in progress. Token put time: 1774459096
2026-03-25 17:18:16 [azure.servicebus._pyamqp.cbs] DEBUG: CBS Put token result (<ManagementExecuteOperationResult.OK: 0>), status code: 202, status_description: b'Accepted'.
2026-03-25 17:18:16 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:16 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACHED: 0> -> <LinkState.ATTACH_SENT: 1>
2026-03-25 17:18:16 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACH_SENT: 1> -> <LinkState.ATTACHED: 3>
2026-03-25 17:18:16 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:16 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:16 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:16 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:16 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item 9810d4ec-5e18-5c77-be21-0e0b402bbf10
2026-03-25 17:18:16 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/faf9fb3a50b44399a16b45e5d0f3641c.jpg>
{'auction_title': '',
'c_o_2_emission_value': 31,
'color': 'Grey',
'currency': 'GBP',
'doors_number': 4,
'emission_standard': 'Euro 6',
'engine': '2L',
'engine_horse_power': 288,
'fuel_type': 'Petrol Plug-in Hybrid',
'id': '9810d4ec-5e18-5c77-be21-0e0b402bbf10',
'is_damaged': False,
'is_operable': True,
'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202603100580971',
'listing': 'directbuy',
'location': 'Leeds, Yorkshire',
'make': 'BMW',
'model': '3 Series',
'odometer': 57323,
'odometer_reading_unit': 'mi',
'origin_country_code': 'GB',
'original_photo_count': 32,
'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_9810d4ec-5e18-5c77-be21-0e0b402bbf10/9810d4ec-5e18-5c77-be21-0e0b402bbf10_image_0.jpg',
'Extension': 'jpg',
'FileSizeInBytes': 227694,
'Height': 1440,
'OriginUrl': 'https://m.atcdn.co.uk/a/media/faf9fb3a50b44399a16b45e5d0f3641c.jpg',
'Width': 1920}],
'price': 19349,
'price_includes_vat': True,
'registration_date': '2021-03-19',
'seats_number': 5,
'seller_name': None,
'title': 'BMW 3 SERIES 2.0 330E M Sport Auto 4dr',
'transmission': 'automatic',
'trim': 'M Sport',
'year': 2021}
2026-03-25 17:18:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603100580971%27 HTTP/1.1" 200 None
2026-03-25 17:18:16 [auto_selling_autotrader.co.uk] INFO: Saving data for 202603100580971: {'created_time': 1774459096.833679, 'last_price_update_time': 1774459096.83369}
2026-03-25 17:18:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202603100580971') HTTP/1.1" 204 0
2026-03-25 17:18:17 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/b1926085dd0c4c22bd7ab6e21b84ad4a.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fcc29bb99d0>
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_d9dc8bf4-72ae-5ef0-80f7-f6c9e7e7c267/d9dc8bf4-72ae-5ef0-80f7-f6c9e7e7c267_image_0.jpg HTTP/1.1" 201 0
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F54f697b60a284893b9de4aecb8103d9e.jpg') HTTP/1.1" 204 0
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/54f697b60a284893b9de4aecb8103d9e.jpg saved to cache
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item d9dc8bf4-72ae-5ef0-80f7-f6c9e7e7c267
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item d9dc8bf4-72ae-5ef0-80f7-f6c9e7e7c267 sending to next pipeline
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fcc246fc310>
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_319b631e-b8f2-5231-b252-77d7b4300540/319b631e-b8f2-5231-b252-77d7b4300540_image_0.jpg HTTP/1.1" 201 0
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252Fe6777e840fa94191ae267cfcf6aae447.jpg') HTTP/1.1" 204 0
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/e6777e840fa94191ae267cfcf6aae447.jpg saved to cache
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 319b631e-b8f2-5231-b252-77d7b4300540
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 319b631e-b8f2-5231-b252-77d7b4300540 sending to next pipeline
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fcc246fe710>
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_381a9612-b895-5616-bdec-cb08a2179d8e/381a9612-b895-5616-bdec-cb08a2179d8e_image_0.jpg HTTP/1.1" 201 0
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F07116b4e0bc240948552cb9891d14b7d.jpg') HTTP/1.1" 204 0
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/07116b4e0bc240948552cb9891d14b7d.jpg saved to cache
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 381a9612-b895-5616-bdec-cb08a2179d8e
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 381a9612-b895-5616-bdec-cb08a2179d8e sending to next pipeline
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fcc2470a9d0>
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_396b41db-eeeb-5ed2-bb8a-d6926295bd20/396b41db-eeeb-5ed2-bb8a-d6926295bd20_image_0.jpg HTTP/1.1" 201 0
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F276275b799ce4049b13d77d02f185700.jpg') HTTP/1.1" 204 0
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/276275b799ce4049b13d77d02f185700.jpg saved to cache
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 396b41db-eeeb-5ed2-bb8a-d6926295bd20
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 396b41db-eeeb-5ed2-bb8a-d6926295bd20 sending to next pipeline
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fcc27584610>
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_717a4521-3f4e-5ef1-abb2-baf793b361d5/717a4521-3f4e-5ef1-abb2-baf793b361d5_image_0.jpg HTTP/1.1" 201 0
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F207d62219ff941b49596bfd214608968.jpg') HTTP/1.1" 204 0
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/207d62219ff941b49596bfd214608968.jpg saved to cache
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 717a4521-3f4e-5ef1-abb2-baf793b361d5
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 717a4521-3f4e-5ef1-abb2-baf793b361d5 sending to next pipeline
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fcc24719090>
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_adbc7c52-da1d-52ca-ba3c-a136f7894af9/adbc7c52-da1d-52ca-ba3c-a136f7894af9_image_0.jpg HTTP/1.1" 201 0
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F8f9352a73cac43d6bfaf45bbcc37f7cd.jpg') HTTP/1.1" 204 0
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/8f9352a73cac43d6bfaf45bbcc37f7cd.jpg saved to cache
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item adbc7c52-da1d-52ca-ba3c-a136f7894af9
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item adbc7c52-da1d-52ca-ba3c-a136f7894af9 sending to next pipeline
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fcc246fd7d0>
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_0a0072cb-85ca-5df5-b1ae-ff4248114fb0/0a0072cb-85ca-5df5-b1ae-ff4248114fb0_image_0.jpg HTTP/1.1" 201 0
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F6f58fb4f11c044148c5921b16b75950b.jpg') HTTP/1.1" 204 0
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/6f58fb4f11c044148c5921b16b75950b.jpg saved to cache
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 0a0072cb-85ca-5df5-b1ae-ff4248114fb0
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 0a0072cb-85ca-5df5-b1ae-ff4248114fb0 sending to next pipeline
2026-03-25 17:18:17 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: d9dc8bf4-72ae-5ef0-80f7-f6c9e7e7c267 with identifier 202603190849828
2026-03-25 17:18:17 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:17 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:17 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item d9dc8bf4-72ae-5ef0-80f7-f6c9e7e7c267
2026-03-25 17:18:17 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/54f697b60a284893b9de4aecb8103d9e.jpg>
{'auction_title': '',
'c_o_2_emission_value': 204,
'color': 'Black',
'currency': 'GBP',
'description': 'Wetbelt Just Done With Genuine Ford Parts \n'
'\n'
'Wide Arch Bodykit\n'
'\n'
'Raptor Grill\n'
'\n'
'Nudge Bar\n'
'\n'
'All Terrain Tyres With Upgraded Alloys\n'
'\n'
'Stunning Truck In & Out\n'
'\n'
'6 Months Warranty\n'
'\n'
'Can Be Upgraded To 3 Years \n'
'\n'
'Competitive Finance Packages Available & Business Finance\n'
'\n'
'Part Exchange Welcome\n'
'\n'
'Existing Finance Can Be Settled',
'doors_number': 2,
'emission_standard': 'Euro 6',
'engine': '2L',
'engine_horse_power': 210,
'fuel_type': 'Diesel',
'id': 'd9dc8bf4-72ae-5ef0-80f7-f6c9e7e7c267',
'is_damaged': False,
'is_operable': True,
'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202603190849828',
'listing': 'directbuy',
'location': 'Leeds, Yorkshire',
'make': 'Ford',
'model': 'Ranger',
'odometer': 63041,
'odometer_reading_unit': 'mi',
'origin_country_code': 'GB',
'original_photo_count': 28,
'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_d9dc8bf4-72ae-5ef0-80f7-f6c9e7e7c267/d9dc8bf4-72ae-5ef0-80f7-f6c9e7e7c267_image_0.jpg',
'Extension': 'jpg',
'FileSizeInBytes': 256631,
'Height': 1440,
'OriginUrl': 'https://m.atcdn.co.uk/a/media/54f697b60a284893b9de4aecb8103d9e.jpg',
'Width': 1920}],
'price': 23988,
'price_includes_vat': True,
'registration_date': '2020-01-28',
'seats_number': 5,
'seller_name': None,
'title': 'Ford 2.0 Ranger Wildtrak EcoBlue 4x4 Auto 4WD',
'transmission': 'automatic',
'trim': 'Wildtrak',
'year': 2020}
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603190849828%27 HTTP/1.1" 200 None
2026-03-25 17:18:17 [auto_selling_autotrader.co.uk] INFO: Saving data for 202603190849828: {'created_time': 1774459097.442873, 'last_price_update_time': 1774459097.442881}
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202603190849828') HTTP/1.1" 204 0
2026-03-25 17:18:17 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: 319b631e-b8f2-5231-b252-77d7b4300540 with identifier 202603100580851
2026-03-25 17:18:17 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:17 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:17 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item 319b631e-b8f2-5231-b252-77d7b4300540
2026-03-25 17:18:17 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/e6777e840fa94191ae267cfcf6aae447.jpg>
{'auction_title': '',
'c_o_2_emission_value': 138,
'color': 'Blue',
'currency': 'GBP',
'doors_number': 4,
'emission_standard': 'Euro 6',
'engine': '1.3L',
'engine_horse_power': 148,
'fuel_type': 'Petrol Hybrid',
'id': '319b631e-b8f2-5231-b252-77d7b4300540',
'is_damaged': False,
'is_operable': True,
'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202603100580851',
'listing': 'directbuy',
'location': 'Leeds, Yorkshire',
'make': 'Mercedes-Benz',
'model': 'CLA',
'odometer': 31841,
'odometer_reading_unit': 'mi',
'origin_country_code': 'GB',
'original_photo_count': 30,
'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_319b631e-b8f2-5231-b252-77d7b4300540/319b631e-b8f2-5231-b252-77d7b4300540_image_0.jpg',
'Extension': 'jpg',
'FileSizeInBytes': 228485,
'Height': 1440,
'OriginUrl': 'https://m.atcdn.co.uk/a/media/e6777e840fa94191ae267cfcf6aae447.jpg',
'Width': 1920}],
'price': 23890,
'price_includes_vat': True,
'registration_date': '2023-07-28',
'seats_number': 5,
'seller_name': None,
'title': 'Mercedes-Benz 1.3 CLA 180 AMG Line Premium Auto 4dr',
'transmission': 'automatic',
'trim': 'AMG Line',
'year': 2023}
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603100580851%27 HTTP/1.1" 200 None
2026-03-25 17:18:17 [auto_selling_autotrader.co.uk] INFO: Saving data for 202603100580851: {'created_time': 1774459097.499168, 'last_price_update_time': 1774459097.499179}
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202603100580851') HTTP/1.1" 204 0
2026-03-25 17:18:17 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: 381a9612-b895-5616-bdec-cb08a2179d8e with identifier 202603190849335
2026-03-25 17:18:17 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:17 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:17 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item 381a9612-b895-5616-bdec-cb08a2179d8e
2026-03-25 17:18:17 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/07116b4e0bc240948552cb9891d14b7d.jpg>
{'auction_title': '',
'c_o_2_emission_value': 138,
'color': 'Black',
'currency': 'GBP',
'description': 'Mint Condition\n'
'\n'
'6 Months Warranty (Can Be Upgraded To 3 Years) \n'
'\n'
'12 Months MOT\n'
'\n'
'Just Serviced\n'
'\n'
'Heated Seats\n'
'\n'
'Reverse Cam\n'
'\n'
'Electric Tailgate\n'
'\n'
'Bigger Screen\n'
'\n'
'Part Exchange Welcome\n'
'\n'
'Competitive Finance Packages Available \n'
'\n'
'Existing Finance Can Be Settled',
'doors_number': 5,
'emission_standard': 'Euro 6',
'engine': '1.6L',
'engine_horse_power': 154,
'fuel_type': 'Petrol',
'id': '381a9612-b895-5616-bdec-cb08a2179d8e',
'is_damaged': False,
'is_operable': True,
'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202603190849335',
'listing': 'directbuy',
'location': 'Leeds, Yorkshire',
'make': 'Mercedes-Benz',
'model': 'GLA',
'odometer': 66000,
'odometer_reading_unit': 'mi',
'origin_country_code': 'GB',
'original_photo_count': 24,
'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_381a9612-b895-5616-bdec-cb08a2179d8e/381a9612-b895-5616-bdec-cb08a2179d8e_image_0.jpg',
'Extension': 'jpg',
'FileSizeInBytes': 215658,
'Height': 1369,
'OriginUrl': 'https://m.atcdn.co.uk/a/media/07116b4e0bc240948552cb9891d14b7d.jpg',
'Width': 1920}],
'price': 12490,
'price_includes_vat': True,
'registration_date': '2017-06-29',
'seats_number': 5,
'seller_name': None,
'title': 'Mercedes-Benz 1.6 GLA 200 AMG Line Executive 5dr',
'transmission': 'manual',
'trim': 'AMG Line',
'year': 2017}
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603190849335%27 HTTP/1.1" 200 None
2026-03-25 17:18:17 [auto_selling_autotrader.co.uk] INFO: Saving data for 202603190849335: {'created_time': 1774459097.547153, 'last_price_update_time': 1774459097.547165}
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202603190849335') HTTP/1.1" 204 0
2026-03-25 17:18:17 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: 396b41db-eeeb-5ed2-bb8a-d6926295bd20 with identifier 202603030380804
2026-03-25 17:18:17 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:17 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:17 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item 396b41db-eeeb-5ed2-bb8a-d6926295bd20
2026-03-25 17:18:17 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/276275b799ce4049b13d77d02f185700.jpg>
{'auction_title': '',
'c_o_2_emission_value': 173,
'color': 'Grey',
'currency': 'GBP',
'description': 'Quantum grey\n'
'\n'
'Heated seats\n'
'\n'
'360 car\n'
'\n'
'Bang & Olufsen Sound System\n'
'\n'
'Cockpit\n'
'\n'
'Black Edition Looks\n'
'\n'
'Audi Service History\n'
'\n'
'2 Keys',
'doors_number': 5,
'emission_standard': 'Euro 6',
'engine': '2L',
'engine_horse_power': 201,
'fuel_type': 'Diesel',
'id': '396b41db-eeeb-5ed2-bb8a-d6926295bd20',
'is_damaged': False,
'is_operable': True,
'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202603030380804',
'listing': 'directbuy',
'location': 'Leeds, Yorkshire',
'make': 'Audi',
'model': 'Q5',
'odometer': 41500,
'odometer_reading_unit': 'mi',
'origin_country_code': 'GB',
'original_photo_count': 12,
'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_396b41db-eeeb-5ed2-bb8a-d6926295bd20/396b41db-eeeb-5ed2-bb8a-d6926295bd20_image_0.jpg',
'Extension': 'jpg',
'FileSizeInBytes': 222358,
'Height': 1440,
'OriginUrl': 'https://m.atcdn.co.uk/a/media/276275b799ce4049b13d77d02f185700.jpg',
'Width': 1920}],
'price': 27490,
'price_includes_vat': True,
'registration_date': '2021-08-26',
'seats_number': 5,
'seller_name': None,
'title': 'Audi 2.0 Q5 S Line 40 TDI MHEV Quattro Semi-Auto 4WD 5dr',
'transmission': 'automatic',
'trim': 'S line',
'year': 2021}
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603030380804%27 HTTP/1.1" 200 None
2026-03-25 17:18:17 [auto_selling_autotrader.co.uk] INFO: Saving data for 202603030380804: {'created_time': 1774459097.608721, 'last_price_update_time': 1774459097.608729}
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202603030380804') HTTP/1.1" 204 0
2026-03-25 17:18:17 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: 717a4521-3f4e-5ef1-abb2-baf793b361d5 with identifier 202603050441201
2026-03-25 17:18:17 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:17 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:17 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item 717a4521-3f4e-5ef1-abb2-baf793b361d5
2026-03-25 17:18:17 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/207d62219ff941b49596bfd214608968.jpg>
{'auction_title': '',
'c_o_2_emission_value': 31,
'color': 'Black',
'currency': 'GBP',
'doors_number': 4,
'emission_standard': 'Euro 6',
'engine': '2L',
'engine_horse_power': 288,
'fuel_type': 'Petrol Plug-in Hybrid',
'id': '717a4521-3f4e-5ef1-abb2-baf793b361d5',
'is_damaged': False,
'is_operable': True,
'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202603050441201',
'listing': 'directbuy',
'location': 'Leeds, Yorkshire',
'make': 'BMW',
'model': '3 Series',
'odometer': 35476,
'odometer_reading_unit': 'mi',
'origin_country_code': 'GB',
'original_photo_count': 38,
'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_717a4521-3f4e-5ef1-abb2-baf793b361d5/717a4521-3f4e-5ef1-abb2-baf793b361d5_image_0.jpg',
'Extension': 'jpg',
'FileSizeInBytes': 231848,
'Height': 1440,
'OriginUrl': 'https://m.atcdn.co.uk/a/media/207d62219ff941b49596bfd214608968.jpg',
'Width': 1920}],
'price': 22190,
'price_includes_vat': True,
'registration_date': '2021-10-04',
'seats_number': 5,
'seller_name': None,
'title': 'BMW 3 SERIES 2.0 330E M Sport Auto 4dr',
'transmission': 'automatic',
'trim': 'M Sport',
'year': 2021}
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603050441201%27 HTTP/1.1" 200 None
2026-03-25 17:18:17 [auto_selling_autotrader.co.uk] INFO: Saving data for 202603050441201: {'created_time': 1774459097.660142, 'last_price_update_time': 1774459097.660152}
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202603050441201') HTTP/1.1" 204 0
2026-03-25 17:18:17 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: adbc7c52-da1d-52ca-ba3c-a136f7894af9 with identifier 202602270283424
2026-03-25 17:18:17 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:17 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:17 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item adbc7c52-da1d-52ca-ba3c-a136f7894af9
2026-03-25 17:18:17 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/8f9352a73cac43d6bfaf45bbcc37f7cd.jpg>
{'auction_title': '',
'c_o_2_emission_value': 152,
'color': 'Black',
'currency': 'GBP',
'description': 'Huge Spec\n'
'\n'
'Harmon Kardon\n'
'\n'
'Heads up Display\n'
'\n'
'Heated Steering Wheel\n'
'\n'
'M Sport Pro\n'
'\n'
'Reverse Cam\n'
'\n'
'Upgraded Wheels',
'doors_number': 4,
'emission_standard': 'Euro 6',
'engine': '2L',
'engine_horse_power': 181,
'fuel_type': 'Petrol',
'id': 'adbc7c52-da1d-52ca-ba3c-a136f7894af9',
'is_damaged': False,
'is_operable': True,
'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202602270283424',
'listing': 'directbuy',
'location': 'Leeds, Yorkshire',
'make': 'BMW',
'model': '3 Series',
'odometer': 30282,
'odometer_reading_unit': 'mi',
'origin_country_code': 'GB',
'original_photo_count': 44,
'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_adbc7c52-da1d-52ca-ba3c-a136f7894af9/adbc7c52-da1d-52ca-ba3c-a136f7894af9_image_0.jpg',
'Extension': 'jpg',
'FileSizeInBytes': 222490,
'Height': 1440,
'OriginUrl': 'https://m.atcdn.co.uk/a/media/8f9352a73cac43d6bfaf45bbcc37f7cd.jpg',
'Width': 1920}],
'price': 25390,
'price_includes_vat': True,
'registration_date': '2021-03-03',
'seats_number': 5,
'seller_name': None,
'title': 'BMW 3 SERIES 2.0 320I M Sport Auto 4dr',
'transmission': 'automatic',
'trim': 'M Sport',
'year': 2021}
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602270283424%27 HTTP/1.1" 200 None
2026-03-25 17:18:17 [auto_selling_autotrader.co.uk] INFO: Saving data for 202602270283424: {'created_time': 1774459097.70501, 'last_price_update_time': 1774459097.705018}
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202602270283424') HTTP/1.1" 204 0
2026-03-25 17:18:17 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: 0a0072cb-85ca-5df5-b1ae-ff4248114fb0 with identifier 202603100581104
2026-03-25 17:18:17 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:17 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:17 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item 0a0072cb-85ca-5df5-b1ae-ff4248114fb0
2026-03-25 17:18:17 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/6f58fb4f11c044148c5921b16b75950b.jpg>
{'auction_title': '',
'c_o_2_emission_value': 36,
'color': 'Black',
'currency': 'GBP',
'doors_number': 4,
'emission_standard': 'Euro 6',
'engine': '2L',
'engine_horse_power': 288,
'fuel_type': 'Petrol Plug-in Hybrid',
'id': '0a0072cb-85ca-5df5-b1ae-ff4248114fb0',
'is_damaged': False,
'is_operable': True,
'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202603100581104',
'listing': 'directbuy',
'location': 'Leeds, Yorkshire',
'make': 'BMW',
'model': '3 Series',
'odometer': 49117,
'odometer_reading_unit': 'mi',
'origin_country_code': 'GB',
'original_photo_count': 40,
'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_0a0072cb-85ca-5df5-b1ae-ff4248114fb0/0a0072cb-85ca-5df5-b1ae-ff4248114fb0_image_0.jpg',
'Extension': 'jpg',
'FileSizeInBytes': 217968,
'Height': 1440,
'OriginUrl': 'https://m.atcdn.co.uk/a/media/6f58fb4f11c044148c5921b16b75950b.jpg',
'Width': 1920}],
'price': 22390,
'price_includes_vat': True,
'registration_date': '2021-06-04',
'seats_number': 5,
'seller_name': None,
'title': 'BMW 3 SERIES 2.0 330E xDrive M Sport Auto 4WD 4dr',
'transmission': 'automatic',
'trim': 'M Sport',
'year': 2021}
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603100581104%27 HTTP/1.1" 200 None
2026-03-25 17:18:17 [auto_selling_autotrader.co.uk] INFO: Saving data for 202603100581104: {'created_time': 1774459097.752567, 'last_price_update_time': 1774459097.752575}
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202603100581104') HTTP/1.1" 204 0
2026-03-25 17:18:17 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/d2fe61618b0b489eb597660898d37fb1.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-25 17:18:17 [scrapy.core.engine] DEBUG: Crawled (200) <POST https://www.autotrader.co.uk/at-graphql> (referer: https://www.autotrader.co.uk/at-gateway?opname=SearchResultsListingsGridQuery&opname=SearchResultsFacetsWithGroupsQuery) ['zyte-api']
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fcc2599da50>
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_aa1919c3-9e71-5e72-acfe-19f3a9a82079/aa1919c3-9e71-5e72-acfe-19f3a9a82079_image_0.jpg HTTP/1.1" 201 0
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252Fb1926085dd0c4c22bd7ab6e21b84ad4a.jpg') HTTP/1.1" 204 0
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/b1926085dd0c4c22bd7ab6e21b84ad4a.jpg saved to cache
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item aa1919c3-9e71-5e72-acfe-19f3a9a82079
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item aa1919c3-9e71-5e72-acfe-19f3a9a82079 sending to next pipeline
2026-03-25 17:18:17 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: aa1919c3-9e71-5e72-acfe-19f3a9a82079 with identifier 202603070485608
2026-03-25 17:18:17 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:17 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:17 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item aa1919c3-9e71-5e72-acfe-19f3a9a82079
2026-03-25 17:18:17 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/b1926085dd0c4c22bd7ab6e21b84ad4a.jpg>
{'auction_title': '',
'c_o_2_emission_value': 0,
'color': 'White',
'currency': 'GBP',
'doors_number': 4,
'emission_standard': 'Euro 6',
'engine_horse_power': 341,
'fuel_type': 'Electric',
'id': 'aa1919c3-9e71-5e72-acfe-19f3a9a82079',
'is_damaged': False,
'is_operable': True,
'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202603070485608',
'listing': 'directbuy',
'location': 'Leeds, Yorkshire',
'make': 'Tesla',
'model': 'Model 3',
'odometer': 32466,
'odometer_reading_unit': 'mi',
'origin_country_code': 'GB',
'original_photo_count': 33,
'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_aa1919c3-9e71-5e72-acfe-19f3a9a82079/aa1919c3-9e71-5e72-acfe-19f3a9a82079_image_0.jpg',
'Extension': 'jpg',
'FileSizeInBytes': 123437,
'Height': 1080,
'OriginUrl': 'https://m.atcdn.co.uk/a/media/b1926085dd0c4c22bd7ab6e21b84ad4a.jpg',
'Width': 1920}],
'price': 20490,
'price_includes_vat': True,
'registration_date': '2021-12-16',
'seats_number': 5,
'seller_name': None,
'title': 'Tesla Model 3 Long Range AWD 4WD 4dr',
'transmission': 'automatic',
'trim': 'Long Range',
'year': 2021}
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603070485608%27 HTTP/1.1" 200 None
2026-03-25 17:18:17 [auto_selling_autotrader.co.uk] INFO: Saving data for 202603070485608: {'created_time': 1774459097.850021, 'last_price_update_time': 1774459097.850028}
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202603070485608') HTTP/1.1" 204 0
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fcc250e8d50>
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-25 17:18:17 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_dddc58a7-570d-5c91-8f47-bb18b640bb46/dddc58a7-570d-5c91-8f47-bb18b640bb46_image_0.jpg HTTP/1.1" 201 0
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252Fd2fe61618b0b489eb597660898d37fb1.jpg') HTTP/1.1" 204 0
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/d2fe61618b0b489eb597660898d37fb1.jpg saved to cache
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item dddc58a7-570d-5c91-8f47-bb18b640bb46
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item dddc58a7-570d-5c91-8f47-bb18b640bb46 sending to next pipeline
2026-03-25 17:18:17 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: dddc58a7-570d-5c91-8f47-bb18b640bb46 with identifier 202602190061263
2026-03-25 17:18:17 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:17 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:17 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item dddc58a7-570d-5c91-8f47-bb18b640bb46
2026-03-25 17:18:17 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/d2fe61618b0b489eb597660898d37fb1.jpg>
{'auction_title': '',
'c_o_2_emission_value': 260,
'color': 'Grey',
'currency': 'GBP',
'description': '2020 70 Porsche Macan Turbo\n'
'\n'
'Crayon Grey\n'
'\n'
'Huge Spec With £25000 Worth Of Extras \n'
'\n'
'Spec List In The Pics\n'
'\n'
'Porsche Embossed Head Rests\n'
'\n'
'Heated & Electric Seats \n'
'\n'
'Pan Roof\n'
'\n'
'Upgraded Wheels\n'
'\n'
'Reverse Cam\n'
'\n'
'Competitive Finance Packages Available\n'
'\n'
'Part Exchange Welcome\n'
'\n'
'Existing Finance Can Be Settled',
'doors_number': 5,
'emission_standard': 'Euro 6',
'engine': '2.9L',
'engine_horse_power': 434,
'fuel_type': 'Petrol',
'id': 'dddc58a7-570d-5c91-8f47-bb18b640bb46',
'is_damaged': False,
'is_operable': True,
'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202602190061263',
'listing': 'directbuy',
'location': 'Leeds, Yorkshire',
'make': 'Porsche',
'model': 'Macan',
'odometer': 46200,
'odometer_reading_unit': 'mi',
'origin_country_code': 'GB',
'original_photo_count': 55,
'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_dddc58a7-570d-5c91-8f47-bb18b640bb46/dddc58a7-570d-5c91-8f47-bb18b640bb46_image_0.jpg',
'Extension': 'jpg',
'FileSizeInBytes': 228179,
'Height': 1440,
'OriginUrl': 'https://m.atcdn.co.uk/a/media/d2fe61618b0b489eb597660898d37fb1.jpg',
'Width': 1920}],
'price': 49990,
'price_includes_vat': True,
'registration_date': '2020-09-01',
'seats_number': 5,
'seller_name': None,
'title': 'Porsche 2.9 Macan T Semi-Auto 4WD 5dr',
'transmission': 'automatic',
'trim': 'Turbo',
'year': 2020}
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602190061263%27 HTTP/1.1" 200 None
2026-03-25 17:18:17 [auto_selling_autotrader.co.uk] INFO: Saving data for 202602190061263: {'created_time': 1774459097.962227, 'last_price_update_time': 1774459097.962236}
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202602190061263') HTTP/1.1" 204 0
2026-03-25 17:18:17 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-25 17:18:17 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: 4c51d92a-c13c-59ea-ac6a-e20187361a27 with identifier: 202602179992097
2026-03-25 17:18:17 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 4c51d92a-c13c-59ea-ac6a-e20187361a27
2026-03-25 17:18:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252Feaefdd43715d4720932aad7a4ebcf701.jpg%27 HTTP/1.1" 200 None
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/eaefdd43715d4720932aad7a4ebcf701.jpg not downloaded yet
2026-03-25 17:18:18 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-25 17:18:18 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: a0316311-a7f9-5b3e-bc9e-0e29b677f14b with identifier: 202602139919458
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item a0316311-a7f9-5b3e-bc9e-0e29b677f14b
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F4e2f0916912946a687542d52884c3348.jpg%27 HTTP/1.1" 200 None
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/4e2f0916912946a687542d52884c3348.jpg not downloaded yet
2026-03-25 17:18:18 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-25 17:18:18 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: 0b9531a4-1909-553d-9b2f-75fccd84afc2 with identifier: 202602129879480
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 0b9531a4-1909-553d-9b2f-75fccd84afc2
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F2cfd4cd2a4624adc94bb6bacde29c3a2.jpg%27 HTTP/1.1" 200 None
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/2cfd4cd2a4624adc94bb6bacde29c3a2.jpg not downloaded yet
2026-03-25 17:18:18 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-25 17:18:18 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: 0d2d0393-1130-58f7-87c2-5ca0192dbd27 with identifier: 202601319572526
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 0d2d0393-1130-58f7-87c2-5ca0192dbd27
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F50747281e3b549769e6db44121bffcc7.jpg%27 HTTP/1.1" 200 None
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/50747281e3b549769e6db44121bffcc7.jpg not downloaded yet
2026-03-25 17:18:18 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-25 17:18:18 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: cfe514d0-682f-52e6-8b77-ed5958e993d7 with identifier: 202512228751456
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item cfe514d0-682f-52e6-8b77-ed5958e993d7
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F315579fd5a5e4d758d312afad1bd4db8.jpg%27 HTTP/1.1" 200 None
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/315579fd5a5e4d758d312afad1bd4db8.jpg not downloaded yet
2026-03-25 17:18:18 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-25 17:18:18 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: 5b2c25d7-acbe-5db7-a5d4-c31db08ec31d with identifier: 202510076951486
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 5b2c25d7-acbe-5db7-a5d4-c31db08ec31d
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F39d8702435f04c0da3ae5eff80d2f8f0.jpg%27 HTTP/1.1" 200 None
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/39d8702435f04c0da3ae5eff80d2f8f0.jpg not downloaded yet
2026-03-25 17:18:18 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/eaefdd43715d4720932aad7a4ebcf701.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-25 17:18:18 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/4e2f0916912946a687542d52884c3348.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-25 17:18:18 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/2cfd4cd2a4624adc94bb6bacde29c3a2.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-25 17:18:18 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/50747281e3b549769e6db44121bffcc7.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-25 17:18:18 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/39d8702435f04c0da3ae5eff80d2f8f0.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fcc24729410>
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_4c51d92a-c13c-59ea-ac6a-e20187361a27/4c51d92a-c13c-59ea-ac6a-e20187361a27_image_0.jpg HTTP/1.1" 201 0
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252Feaefdd43715d4720932aad7a4ebcf701.jpg') HTTP/1.1" 204 0
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/eaefdd43715d4720932aad7a4ebcf701.jpg saved to cache
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 4c51d92a-c13c-59ea-ac6a-e20187361a27
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 4c51d92a-c13c-59ea-ac6a-e20187361a27 sending to next pipeline
2026-03-25 17:18:18 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: 4c51d92a-c13c-59ea-ac6a-e20187361a27 with identifier 202602179992097
2026-03-25 17:18:18 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:18 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:18 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item 4c51d92a-c13c-59ea-ac6a-e20187361a27
2026-03-25 17:18:18 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/eaefdd43715d4720932aad7a4ebcf701.jpg>
{'auction_title': '',
'c_o_2_emission_value': 31,
'color': 'Grey',
'currency': 'GBP',
'description': 'Upgraded Gloss Black M Sport Pro Alloys\n'
'\n'
'Skyscraper Grey\n'
'\n'
'MP Kit \n'
'\n'
'LCI Facelift Model\n'
'\n'
'Big Screen Displays\n'
'\n'
'Heated Seats\n'
'\n'
'Reverse Cam \n'
'\n'
'Full Service History\n'
'\n'
'6 Months Warranty As Standard, Can Be Upgraded To 3 Years\n'
'\n'
'Part Exchange Welcome\n'
'\n'
'Existing Finance Can Be Settled\n'
'\n'
'Competitive Finance Packages Available',
'doors_number': 4,
'emission_standard': 'Euro 6',
'engine': '2L',
'engine_horse_power': 288,
'fuel_type': 'Petrol Plug-in Hybrid',
'id': '4c51d92a-c13c-59ea-ac6a-e20187361a27',
'is_damaged': False,
'is_operable': True,
'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202602179992097',
'listing': 'directbuy',
'location': 'Leeds, Yorkshire',
'make': 'BMW',
'model': '3 Series',
'odometer': 69510,
'odometer_reading_unit': 'mi',
'origin_country_code': 'GB',
'original_photo_count': 43,
'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_4c51d92a-c13c-59ea-ac6a-e20187361a27/4c51d92a-c13c-59ea-ac6a-e20187361a27_image_0.jpg',
'Extension': 'jpg',
'FileSizeInBytes': 226294,
'Height': 1440,
'OriginUrl': 'https://m.atcdn.co.uk/a/media/eaefdd43715d4720932aad7a4ebcf701.jpg',
'Width': 1920}],
'price': 23490,
'price_includes_vat': True,
'registration_date': '2023-01-31',
'seats_number': 5,
'seller_name': None,
'title': 'BMW 3 SERIES 2.0 330E M Sport Auto 4dr',
'transmission': 'automatic',
'trim': 'M Sport',
'year': 2023}
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602179992097%27 HTTP/1.1" 200 None
2026-03-25 17:18:18 [auto_selling_autotrader.co.uk] INFO: Saving data for 202602179992097: {'created_time': 1774459098.331269, 'last_price_update_time': 1774459098.331278}
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202602179992097') HTTP/1.1" 204 0
2026-03-25 17:18:18 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/315579fd5a5e4d758d312afad1bd4db8.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fcc24732490>
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_a0316311-a7f9-5b3e-bc9e-0e29b677f14b/a0316311-a7f9-5b3e-bc9e-0e29b677f14b_image_0.jpg HTTP/1.1" 201 0
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F4e2f0916912946a687542d52884c3348.jpg') HTTP/1.1" 204 0
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/4e2f0916912946a687542d52884c3348.jpg saved to cache
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item a0316311-a7f9-5b3e-bc9e-0e29b677f14b
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item a0316311-a7f9-5b3e-bc9e-0e29b677f14b sending to next pipeline
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fcc24735510>
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_0b9531a4-1909-553d-9b2f-75fccd84afc2/0b9531a4-1909-553d-9b2f-75fccd84afc2_image_0.jpg HTTP/1.1" 201 0
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F2cfd4cd2a4624adc94bb6bacde29c3a2.jpg') HTTP/1.1" 204 0
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/2cfd4cd2a4624adc94bb6bacde29c3a2.jpg saved to cache
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 0b9531a4-1909-553d-9b2f-75fccd84afc2
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 0b9531a4-1909-553d-9b2f-75fccd84afc2 sending to next pipeline
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fcc246fdf50>
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_0d2d0393-1130-58f7-87c2-5ca0192dbd27/0d2d0393-1130-58f7-87c2-5ca0192dbd27_image_0.jpg HTTP/1.1" 201 0
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F50747281e3b549769e6db44121bffcc7.jpg') HTTP/1.1" 204 0
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/50747281e3b549769e6db44121bffcc7.jpg saved to cache
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 0d2d0393-1130-58f7-87c2-5ca0192dbd27
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 0d2d0393-1130-58f7-87c2-5ca0192dbd27 sending to next pipeline
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fcc27516d90>
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_5b2c25d7-acbe-5db7-a5d4-c31db08ec31d/5b2c25d7-acbe-5db7-a5d4-c31db08ec31d_image_0.jpg HTTP/1.1" 201 0
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F39d8702435f04c0da3ae5eff80d2f8f0.jpg') HTTP/1.1" 204 0
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/39d8702435f04c0da3ae5eff80d2f8f0.jpg saved to cache
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 5b2c25d7-acbe-5db7-a5d4-c31db08ec31d
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 5b2c25d7-acbe-5db7-a5d4-c31db08ec31d sending to next pipeline
2026-03-25 17:18:18 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: a0316311-a7f9-5b3e-bc9e-0e29b677f14b with identifier 202602139919458
2026-03-25 17:18:18 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:18 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:18 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item a0316311-a7f9-5b3e-bc9e-0e29b677f14b
2026-03-25 17:18:18 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/4e2f0916912946a687542d52884c3348.jpg>
{'auction_title': '',
'c_o_2_emission_value': 0,
'color': 'White',
'currency': 'GBP',
'doors_number': 5,
'emission_standard': 'Euro 6',
'engine_horse_power': 201,
'fuel_type': 'Electric',
'id': 'a0316311-a7f9-5b3e-bc9e-0e29b677f14b',
'is_damaged': False,
'is_operable': True,
'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202602139919458',
'listing': 'directbuy',
'location': 'Leeds, Yorkshire',
'make': 'Volkswagen',
'model': 'ID.3',
'odometer': 52800,
'odometer_reading_unit': 'mi',
'origin_country_code': 'GB',
'original_photo_count': 24,
'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_a0316311-a7f9-5b3e-bc9e-0e29b677f14b/a0316311-a7f9-5b3e-bc9e-0e29b677f14b_image_0.jpg',
'Extension': 'jpg',
'FileSizeInBytes': 217866,
'Height': 1440,
'OriginUrl': 'https://m.atcdn.co.uk/a/media/4e2f0916912946a687542d52884c3348.jpg',
'Width': 1920}],
'price': 11890,
'price_includes_vat': True,
'registration_date': '2021-05-14',
'seats_number': 5,
'seller_name': None,
'title': 'Volkswagen ID.3 ID3 Life 5dr',
'transmission': 'automatic',
'trim': 'Life',
'year': 2021}
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602139919458%27 HTTP/1.1" 200 None
2026-03-25 17:18:18 [auto_selling_autotrader.co.uk] INFO: Saving data for 202602139919458: {'created_time': 1774459098.544052, 'last_price_update_time': 1774459098.544059}
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202602139919458') HTTP/1.1" 204 0
2026-03-25 17:18:18 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: 0b9531a4-1909-553d-9b2f-75fccd84afc2 with identifier 202602129879480
2026-03-25 17:18:18 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:18 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:18 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item 0b9531a4-1909-553d-9b2f-75fccd84afc2
2026-03-25 17:18:18 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/2cfd4cd2a4624adc94bb6bacde29c3a2.jpg>
{'auction_title': '',
'c_o_2_emission_value': 31,
'color': 'Black',
'currency': 'GBP',
'description': '1 Owner From New\n'
'\n'
'MP Bodykit \n'
'\n'
'Facelift LCI Model\n'
'\n'
'Upgraded 19" Alloys\n'
'\n'
'Big Screen Displays\n'
'\n'
'Alcantara Seats \n'
'\n'
'Heated Seats\n'
'\n'
'Car Play\n'
'\n'
'Reverse Cam\n'
'\n'
'2 Keys\n'
'\n'
'Full Service History\n'
'\n'
'Competitive Finance Packages Available \n'
'\n'
'Existing Finance Can Be Settled, Part Exchange Welcome\n'
'\n'
'6 Months Warranty As Standard, Can Be Upgraded to 3 Years',
'doors_number': 5,
'emission_standard': 'Euro 6',
'engine': '2L',
'engine_horse_power': 288,
'fuel_type': 'Petrol Plug-in Hybrid',
'id': '0b9531a4-1909-553d-9b2f-75fccd84afc2',
'is_damaged': False,
'is_operable': True,
'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202602129879480',
'listing': 'directbuy',
'location': 'Leeds, Yorkshire',
'make': 'BMW',
'model': '3 Series',
'odometer': 59222,
'odometer_reading_unit': 'mi',
'origin_country_code': 'GB',
'original_photo_count': 48,
'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_0b9531a4-1909-553d-9b2f-75fccd84afc2/0b9531a4-1909-553d-9b2f-75fccd84afc2_image_0.jpg',
'Extension': 'jpg',
'FileSizeInBytes': 197189,
'Height': 1440,
'OriginUrl': 'https://m.atcdn.co.uk/a/media/2cfd4cd2a4624adc94bb6bacde29c3a2.jpg',
'Width': 1920}],
'price': 23790,
'price_includes_vat': True,
'registration_date': '2022-10-31',
'seats_number': 5,
'seller_name': None,
'title': 'BMW 3 SERIES 2.0 330E M Sport Auto 5dr',
'transmission': 'automatic',
'trim': 'M Sport',
'year': 2022}
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602129879480%27 HTTP/1.1" 200 None
2026-03-25 17:18:18 [auto_selling_autotrader.co.uk] INFO: Saving data for 202602129879480: {'created_time': 1774459098.589153, 'last_price_update_time': 1774459098.589159}
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202602129879480') HTTP/1.1" 204 0
2026-03-25 17:18:18 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: 0d2d0393-1130-58f7-87c2-5ca0192dbd27 with identifier 202601319572526
2026-03-25 17:18:18 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:18 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:18 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item 0d2d0393-1130-58f7-87c2-5ca0192dbd27
2026-03-25 17:18:18 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/50747281e3b549769e6db44121bffcc7.jpg>
{'auction_title': '',
'c_o_2_emission_value': 23,
'color': 'Blue',
'currency': 'GBP',
'description': 'Upgraded Alloys\n'
'\n'
'CLA45 Style Diffuser With Quad Exhaust\n'
'\n'
'Ambient Lighting \n'
'\n'
'GTS Grill\n'
'\n'
'Side Skirts\n'
'\n'
'Spoiler',
'doors_number': 4,
'emission_standard': 'Euro 6',
'engine': '1.3L',
'engine_horse_power': 215,
'fuel_type': 'Petrol Plug-in Hybrid',
'id': '0d2d0393-1130-58f7-87c2-5ca0192dbd27',
'is_damaged': False,
'is_operable': True,
'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202601319572526',
'listing': 'directbuy',
'location': 'Leeds, Yorkshire',
'make': 'Mercedes-Benz',
'model': 'CLA',
'odometer': 20923,
'odometer_reading_unit': 'mi',
'origin_country_code': 'GB',
'original_photo_count': 44,
'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_0d2d0393-1130-58f7-87c2-5ca0192dbd27/0d2d0393-1130-58f7-87c2-5ca0192dbd27_image_0.jpg',
'Extension': 'jpg',
'FileSizeInBytes': 235781,
'Height': 1440,
'OriginUrl': 'https://m.atcdn.co.uk/a/media/50747281e3b549769e6db44121bffcc7.jpg',
'Width': 1920}],
'price': 23249,
'price_includes_vat': True,
'registration_date': '2021-12-10',
'seats_number': 5,
'seller_name': None,
'title': 'Mercedes-Benz 1.3 CLA 250 AMG Line Premium E Auto 4dr',
'transmission': 'automatic',
'trim': 'AMG Line',
'year': 2021}
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202601319572526%27 HTTP/1.1" 200 None
2026-03-25 17:18:18 [auto_selling_autotrader.co.uk] INFO: Saving data for 202601319572526: {'created_time': 1774459098.639145, 'last_price_update_time': 1774459098.639154}
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202601319572526') HTTP/1.1" 204 0
2026-03-25 17:18:18 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: 5b2c25d7-acbe-5db7-a5d4-c31db08ec31d with identifier 202510076951486
2026-03-25 17:18:18 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:18 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:18 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item 5b2c25d7-acbe-5db7-a5d4-c31db08ec31d
2026-03-25 17:18:18 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/39d8702435f04c0da3ae5eff80d2f8f0.jpg>
{'auction_title': '',
'c_o_2_emission_value': 179,
'color': 'Black',
'currency': 'GBP',
'description': 'Huge Spec X5 M50D\n'
'\n'
'Sky Lounge Panoramic Roof\n'
'\n'
'Carbon Black\n'
'\n'
'Laser Lights\n'
'\n'
'Visibility Pack\n'
'\n'
'Heads Up Display\n'
'\n'
'BMW Individual Extended Merino Leather\n'
'\n'
'BMW Individual Instrument Panel In Leather\n'
'\n'
'Comfort Package\n'
'\n'
'Technology\n'
'\n'
'Harmon Kardon\n'
'\n'
'360 Camera\n'
'\n'
'Adaptive 2 Axle Air Suspension \n'
'\n'
'& More \n'
'\n'
'Competitive Finance Packages Available \n'
'\n'
'6 Months Warranty Can Be Extended Upto 3 Years \n'
'\n'
'Part Exchange Welcome & Existing Finance Can Be Settled',
'doors_number': 5,
'emission_standard': 'Euro 6',
'engine': '3L',
'engine_horse_power': 395,
'fuel_type': 'Diesel',
'id': '5b2c25d7-acbe-5db7-a5d4-c31db08ec31d',
'is_damaged': False,
'is_operable': True,
'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202510076951486',
'listing': 'directbuy',
'location': 'Leeds, Yorkshire',
'make': 'BMW',
'model': 'X5',
'odometer': 85879,
'odometer_reading_unit': 'mi',
'origin_country_code': 'GB',
'original_photo_count': 46,
'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_5b2c25d7-acbe-5db7-a5d4-c31db08ec31d/5b2c25d7-acbe-5db7-a5d4-c31db08ec31d_image_0.jpg',
'Extension': 'jpg',
'FileSizeInBytes': 218282,
'Height': 1440,
'OriginUrl': 'https://m.atcdn.co.uk/a/media/39d8702435f04c0da3ae5eff80d2f8f0.jpg',
'Width': 1920}],
'price': 36890,
'price_includes_vat': True,
'registration_date': '2019-01-18',
'seats_number': 5,
'seller_name': None,
'title': 'BMW 3.0 X5 M50D Auto 4WD 5dr',
'transmission': 'automatic',
'year': 2019}
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202510076951486%27 HTTP/1.1" 200 None
2026-03-25 17:18:18 [auto_selling_autotrader.co.uk] INFO: Saving data for 202510076951486: {'created_time': 1774459098.684849, 'last_price_update_time': 1774459098.684855}
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202510076951486') HTTP/1.1" 204 0
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fcc2598c650>
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-25 17:18:18 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_cfe514d0-682f-52e6-8b77-ed5958e993d7/cfe514d0-682f-52e6-8b77-ed5958e993d7_image_0.jpg HTTP/1.1" 201 0
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F315579fd5a5e4d758d312afad1bd4db8.jpg') HTTP/1.1" 204 0
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/315579fd5a5e4d758d312afad1bd4db8.jpg saved to cache
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item cfe514d0-682f-52e6-8b77-ed5958e993d7
2026-03-25 17:18:18 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item cfe514d0-682f-52e6-8b77-ed5958e993d7 sending to next pipeline
2026-03-25 17:18:18 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: cfe514d0-682f-52e6-8b77-ed5958e993d7 with identifier 202512228751456
2026-03-25 17:18:18 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:18 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:18 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item cfe514d0-682f-52e6-8b77-ed5958e993d7
2026-03-25 17:18:18 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/315579fd5a5e4d758d312afad1bd4db8.jpg>
{'auction_title': '',
'c_o_2_emission_value': 146,
'color': 'Grey',
'currency': 'GBP',
'description': '1 Owner\n'
'\n'
'Ambient Lighting\n'
'\n'
'Sound System\n'
'\n'
'Reverse Cam\n'
'\n'
'Stunning Inside & Out\n'
'\n'
'GTS Grill\n'
'\n'
'Upgraded Alloys \n'
'\n'
'Rear Spoiler \n'
'\n'
'Full Heated Leather\n'
'\n'
'Big Screens',
'doors_number': 4,
'emission_standard': 'Euro 6',
'engine': '1.5L',
'engine_horse_power': 221,
'fuel_type': 'Petrol Hybrid',
'id': 'cfe514d0-682f-52e6-8b77-ed5958e993d7',
'is_damaged': False,
'is_operable': True,
'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202512228751456',
'listing': 'directbuy',
'location': 'Leeds, Yorkshire',
'make': 'Mercedes-Benz',
'model': 'C Class',
'odometer': 25988,
'odometer_reading_unit': 'mi',
'origin_country_code': 'GB',
'original_photo_count': 46,
'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_cfe514d0-682f-52e6-8b77-ed5958e993d7/cfe514d0-682f-52e6-8b77-ed5958e993d7_image_0.jpg',
'Extension': 'jpg',
'FileSizeInBytes': 199641,
'Height': 1440,
'OriginUrl': 'https://m.atcdn.co.uk/a/media/315579fd5a5e4d758d312afad1bd4db8.jpg',
'Width': 1920}],
'price': 27400,
'price_includes_vat': True,
'registration_date': '2023-07-21',
'seats_number': 5,
'seller_name': None,
'title': 'Mercedes-Benz C-Class 1.5 C 200 AMG Line MHEV Auto 4dr',
'transmission': 'automatic',
'trim': 'AMG Line',
'year': 2023}
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202512228751456%27 HTTP/1.1" 200 None
2026-03-25 17:18:18 [auto_selling_autotrader.co.uk] INFO: Saving data for 202512228751456: {'created_time': 1774459098.785542, 'last_price_update_time': 1774459098.785549}
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202512228751456') HTTP/1.1" 204 0
2026-03-25 17:18:18 [scrapy.core.engine] INFO: Closing spider (finished)
2026-03-25 17:18:18 [crawlers.pipelines.save_to_cdlq_db_auto_selling] DEBUG: [SaveToCdlqDbAutoSelling] failed to close cursor: 'SaveToCdlqDbAutoSelling' object has no attribute 'cursor'
2026-03-25 17:18:18 [crawlers.pipelines.save_to_cdlq_db_auto_selling] DEBUG: [SaveToCdlqDbAutoSelling] failed to close connection: 'SaveToCdlqDbAutoSelling' object has no attribute 'conn'
2026-03-25 17:18:18 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:18 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 17:18:18 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process scrape complete CloudEvent for item 94508962286e11f18f76c264925fbcfa
2026-03-25 17:18:18 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2026-03-25 17:18:18 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2026-03-25 17:18:18 [azure.servicebus._pyamqp.management_link] INFO: Management link receiver state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2026-03-25 17:18:18 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2026-03-25 17:18:18 [azure.servicebus._pyamqp.management_link] INFO: Management link sender state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2026-03-25 17:18:18 [azure.servicebus._pyamqp.session] INFO: Session state changed: <SessionState.MAPPED: 3> -> <SessionState.END_SENT: 4>
2026-03-25 17:18:18 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.OPENED: 9> -> <ConnectionState.CLOSE_SENT: 11>
2026-03-25 17:18:18 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.CLOSE_SENT: 11> -> <ConnectionState.END: 13>
2026-03-25 17:18:18 [azure.servicebus._pyamqp.session] INFO: Session state changed: <SessionState.END_SENT: 4> -> <SessionState.DISCARDING: 6>
2026-03-25 17:18:18 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2026-03-25 17:18:18 [azure.servicebus._pyamqp.management_link] INFO: Management link sender state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2026-03-25 17:18:18 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2026-03-25 17:18:18 [azure.servicebus._pyamqp.management_link] INFO: Management link receiver state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2026-03-25 17:18:18 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2026-03-25 17:18:19 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: [AzureBusPipeline] connection closed
2026-03-25 17:18:19 [auto_selling_autotrader.co.uk] INFO: auto_selling_autotrader.co.uk Crawl ended with reason finished, scrape types: {<ScrapeType.NEW: 1>: 0, <ScrapeType.NEW_DUPLICATE_ID: 4>: 0, <ScrapeType.PRICE_UPDATE: 2>: 0, <ScrapeType.AUCTION_UPDATE: 3>: 0, <ScrapeType.SKIPPED: 0>: 0}
2026-03-25 17:18:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 17:18:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20last_price_update_time%20lt%201773163099 HTTP/1.1" 200 None
2026-03-25 17:18:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "DELETE /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202602210115911') HTTP/1.1" 204 0
2026-03-25 17:18:19 [scrapy.extensions.feedexport] INFO: Stored jsonlines feed (16 items) in: file:///var/lib/scrapyd/items/sourcing_v2/auto_selling_autotrader.co.uk/94508962286e11f18f76c264925fbcfa.jl
2026-03-25 17:18:19 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 128559,
'downloader/request_count': 19,
'downloader/request_method_count/GET': 16,
'downloader/request_method_count/POST': 3,
'downloader/response_bytes': 4320721,
'downloader/response_count': 19,
'downloader/response_status_count/200': 19,
'elapsed_time_seconds': 15.022189,
'feedexport/success_count/FileFeedStorage': 1,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2026, 3, 25, 17, 18, 19, 819646, tzinfo=datetime.timezone.utc),
'item_scraped_count': 16,
'log_count/DEBUG': 305,
'log_count/INFO': 288,
'memusage/max': 147460096,
'memusage/startup': 147460096,
'photo_download_count': 16,
'request_depth_max': 2,
'response_received_count': 3,
'scheduler/dequeued': 19,
'scheduler/dequeued/memory': 19,
'scheduler/enqueued': 19,
'scheduler/enqueued/memory': 19,
'scrape_type/new': 16,
'scrapy-zyte-api/429': 0,
'scrapy-zyte-api/attempts': 3,
'scrapy-zyte-api/error_ratio': 0.0,
'scrapy-zyte-api/errors': 0,
'scrapy-zyte-api/fatal_errors': 0,
'scrapy-zyte-api/mean_connection_seconds': 3.3048209291882813,
'scrapy-zyte-api/mean_response_seconds': 3.6251200640884536,
'scrapy-zyte-api/processed': 3,
'scrapy-zyte-api/request_args/customHttpRequestHeaders': 3,
'scrapy-zyte-api/request_args/experimental.requestCookies': 2,
'scrapy-zyte-api/request_args/experimental.responseCookies': 3,
'scrapy-zyte-api/request_args/httpRequestBody': 3,
'scrapy-zyte-api/request_args/httpRequestMethod': 3,
'scrapy-zyte-api/request_args/httpResponseBody': 3,
'scrapy-zyte-api/request_args/httpResponseHeaders': 3,
'scrapy-zyte-api/request_args/url': 3,
'scrapy-zyte-api/status_codes/200': 3,
'scrapy-zyte-api/success': 3,
'scrapy-zyte-api/success_ratio': 1.0,
'scrapy-zyte-api/throttle_ratio': 0.0,
'start_time': datetime.datetime(2026, 3, 25, 17, 18, 4, 797457, tzinfo=datetime.timezone.utc)}
2026-03-25 17:18:19 [scrapy.core.engine] INFO: Spider closed (finished)
2026-03-25 17:18:19 [asyncio] ERROR: Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7fcc2914bb50>
2026-03-25 17:18:19 [asyncio] ERROR: Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7fcc291583d0>