2026-03-30 12:11:32 [scrapy.utils.log] INFO: Scrapy 2.11.2 started (bot: SourcingV2)
2026-03-30 12:11:32 [scrapy.utils.log] INFO: Versions: lxml 5.2.2.0, libxml2 2.12.6, cssselect 1.2.0, parsel 1.9.1, w3lib 2.1.2, Twisted 24.3.0, Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0], pyOpenSSL 24.1.0 (OpenSSL 3.2.1 30 Jan 2024), cryptography 42.0.7, Platform Linux-5.15.0-1098-azure-x86_64-with-glibc2.36
2026-03-30 12:11:32 [auto_selling_autotrader.co.uk] INFO: Starting auto_selling_autotrader.co.uk spider
2026-03-30 12:11:32 [auto_selling_autotrader.co.uk] INFO: Quick mode enabled: batching detail requests in groups of 10
2026-03-30 12:11:32 [auto_selling_autotrader.co.uk] INFO: Limiting photo downloads to 1 per item
2026-03-30 12:11:32 [auto_selling_autotrader.co.uk] INFO: _job: 93ca90c02c3111f1b320c264925fbcfa, SCRAPY_JOB: None
2026-03-30 12:11:32 [scrapy.addons] INFO: Enabled addons:
[]
2026-03-30 12:11:32 [asyncio] DEBUG: Using selector: EpollSelector
2026-03-30 12:11:32 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.asyncioreactor.AsyncioSelectorReactor
2026-03-30 12:11:32 [scrapy.utils.log] DEBUG: Using asyncio event loop: asyncio.unix_events._UnixSelectorEventLoop
2026-03-30 12:11:32 [scrapy.extensions.telnet] INFO: Telnet Password: f1c985082da9cb86
2026-03-30 12:11:33 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.memusage.MemoryUsage',
 'scrapy.extensions.feedexport.FeedExporter',
 'scrapy.extensions.logstats.LogStats',
 'scrapy.extensions.closespider.CloseSpider']
2026-03-30 12:11:33 [scrapy.crawler] INFO: Overridden settings:
{'BOT_NAME': 'SourcingV2',
 'CLOSESPIDER_TIMEOUT': 3600,
 'DOWNLOAD_MAXSIZE': 52428800,
 'DOWNLOAD_WARNSIZE': 10485760,
 'FEED_EXPORT_ENCODING': 'utf-8',
 'LOG_FILE': '/var/log/scrapyd/logs/sourcing_v2/auto_selling_autotrader.co.uk/93ca90c02c3111f1b320c264925fbcfa.log',
 'LOG_FORMATTER': 'crawlers.log_formatter.SourcingLogFormatter',
 'MEMUSAGE_LIMIT_MB': 2048,
 'MEMUSAGE_WARNING_MB': 1536,
 'NEWSPIDER_MODULE': 'spiders',
 'REQUEST_FINGERPRINTER_CLASS': 'scrapy_zyte_api.ScrapyZyteAPIRequestFingerprinter',
 'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7',
 'SPIDER_MODULES': ['spiders', 'auth_check'],
 'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor',
 'USER_AGENT': ''}
2026-03-30 12:11:33 [scrapy_zyte_api.handler] INFO: Using a Zyte API key starting with '5857011'
2026-03-30 12:11:33 [scrapy_zyte_api.handler] INFO: Using a Zyte API key starting with '5857011'
2026-03-30 12:11:33 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
 'scrapy_zyte_api.ScrapyZyteAPIDownloaderMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
 'scrapy.downloadermiddlewares.stats.DownloaderStats']
2026-03-30 12:11:33 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Setting up IdGenerationMiddleware for auto_selling
2026-03-30 12:11:33 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
 'scrapy_zyte_api.ScrapyZyteAPISpiderMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware',
 'crawlers.middlewares.photo_download_middleware.PhotoDownloadMiddleware',
 'crawlers.auto_selling.middlewares.id_gen_middleware.IdGenMiddleware']
2026-03-30 12:11:33 [scrapy.middleware] INFO: Enabled item pipelines:
['crawlers.pipelines.post_to_azurebus_auto_selling.AzureBusPipelineAutoSelling',
 'crawlers.pipelines.save_to_cdlq_db_auto_selling.SaveToCdlqDbAutoSelling']
2026-03-30 12:11:33 [scrapy.core.engine] INFO: Spider opened
2026-03-30 12:11:33 [azure.identity._credentials.environment] INFO: Incomplete environment configuration for EnvironmentCredential. These variables are set: AZURE_CLIENT_ID, AZURE_TENANT_ID
2026-03-30 12:11:33 [common.service_bus.service_bus_client_factory] INFO: ServiceBusClient initialized via DefaultAzureCredential for alx-production-bus.servicebus.windows.net
2026-03-30 12:11:33 [crawlers.pipelines.base_azure_bus_pipeline] INFO: [AzureBusPipeline] Connected to topic auto.sourcing-v2-quickmode
2026-03-30 12:11:33 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-03-30 12:11:33 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6025
2026-03-30 12:11:33 [scrapy.extensions.memusage] INFO: Peak memory usage is 140MiB
2026-03-30 12:11:33 [scrapy-playwright] INFO: Starting download handler
2026-03-30 12:11:33 [scrapy-playwright] INFO: Starting download handler
2026-03-30 12:11:38 [auto_selling_autotrader.co.uk] INFO: Scraper started with retailer ID: 10026409 and multipost_profile_id: 1aa21d5d-d156-4390-8fa7-daefcf8c7896 and job_id: 93ca90c02c3111f1b320c264925fbcfa and url: https://www.autotrader.co.uk/dealers/copnor-road-cars-10026409
2026-03-30 12:11:38 [auto_selling_autotrader.co.uk] INFO: Getting list for retailer ID: 10026409, page: 1
2026-03-30 12:11:38 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 1st time calling it.
2026-03-30 12:11:58 [scrapy.core.engine] DEBUG: Crawled (200) <POST https://www.autotrader.co.uk/at-gateway?opname=SearchResultsListingsGridQuery&opname=SearchResultsFacetsWithGroupsQuery> (referer: None) ['zyte-api']
2026-03-30 12:11:58 [auto_selling_autotrader.co.uk] DEBUG: response: <200 https://www.autotrader.co.uk/at-gateway?opname=SearchResultsListingsGridQuery&opname=SearchResultsFacetsWithGroupsQuery>
2026-03-30 12:11:58 [auto_selling_autotrader.co.uk] INFO: Page: 1/1, total results: 15
2026-03-30 12:11:58 [azure.identity._credentials.environment] INFO: Incomplete environment configuration for EnvironmentCredential. These variables are set: AZURE_CLIENT_ID, AZURE_TENANT_ID
2026-03-30 12:11:58 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 2118
2026-03-30 12:11:59 [azure.identity._credentials.chained] INFO: DefaultAzureCredential acquired a token from WorkloadIdentityCredential
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxsourcingstorageprod.table.core.windows.net:443
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603180791647%27 HTTP/1.1" 200 None
2026-03-30 12:11:59 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202603180791647: 1
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603160740452%27 HTTP/1.1" 200 None
2026-03-30 12:11:59 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202603160740452: 1
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603040405261%27 HTTP/1.1" 200 None
2026-03-30 12:11:59 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202603040405261: 1
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603040386452%27 HTTP/1.1" 200 None
2026-03-30 12:11:59 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202603040386452: 1
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603020320093%27 HTTP/1.1" 200 None
2026-03-30 12:11:59 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202603020320093: 1
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602250206482%27 HTTP/1.1" 200 None
2026-03-30 12:11:59 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202602250206482: 1
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602200085700%27 HTTP/1.1" 200 None
2026-03-30 12:11:59 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202602200085700: 1
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602200074354%27 HTTP/1.1" 200 None
2026-03-30 12:11:59 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202602200074354: 1
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602190052151%27 HTTP/1.1" 200 None
2026-03-30 12:11:59 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202602190052151: 1
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602169963132%27 HTTP/1.1" 200 None
2026-03-30 12:11:59 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202602169963132: 1
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602169955425%27 HTTP/1.1" 200 None
2026-03-30 12:11:59 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202602169955425: 1
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602129872004%27 HTTP/1.1" 200 None
2026-03-30 12:11:59 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202602129872004: 1
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202601259405817%27 HTTP/1.1" 200 None
2026-03-30 12:11:59 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202601259405817: 1
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202601179216399%27 HTTP/1.1" 200 None
2026-03-30 12:11:59 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202601179216399: 1
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:11:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202601159154661%27 HTTP/1.1" 200 None
2026-03-30 12:11:59 [auto_selling_autotrader.co.uk] INFO: Scrape type for 202601159154661: 1
2026-03-30 12:11:59 [auto_selling_autotrader.co.uk] INFO: Quick mode: batching 10 detail requests (batch 1)
2026-03-30 12:11:59 [auto_selling_autotrader.co.uk] INFO: Quick mode: batching 5 detail requests (batch 2)
2026-03-30 12:11:59 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 1st time calling it.
2026-03-30 12:11:59 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 1st time calling it.
2026-03-30 12:12:00 [scrapy.core.engine] DEBUG: Crawled (200) <POST https://www.autotrader.co.uk/at-graphql> (referer: https://www.autotrader.co.uk/at-gateway?opname=SearchResultsListingsGridQuery&opname=SearchResultsFacetsWithGroupsQuery) ['zyte-api']
2026-03-30 12:12:00 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-30 12:12:00 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: 3c8f2bee-3be6-500b-b537-3e2e2bb27b10 with identifier: 202602169955425
2026-03-30 12:12:00 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 3c8f2bee-3be6-500b-b537-3e2e2bb27b10
2026-03-30 12:12:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252Fb48768ccd35043ebba6fcc56b6a4665b.jpg%27 HTTP/1.1" 200 None
2026-03-30 12:12:00 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/b48768ccd35043ebba6fcc56b6a4665b.jpg not downloaded yet
2026-03-30 12:12:00 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-30 12:12:00 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: f5eaaaa7-6b74-598c-9001-7f99e69425c2 with identifier: 202602129872004
2026-03-30 12:12:00 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item f5eaaaa7-6b74-598c-9001-7f99e69425c2
2026-03-30 12:12:00 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F77cc393b9a1842a8aab2c8b83b60ec23.jpg%27 HTTP/1.1" 200 None
2026-03-30 12:12:01 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/77cc393b9a1842a8aab2c8b83b60ec23.jpg not downloaded yet
2026-03-30 12:12:01 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-30 12:12:01 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: a509fd1a-780f-5f24-9ea2-df648330fc4e with identifier: 202601259405817
2026-03-30 12:12:01 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item a509fd1a-780f-5f24-9ea2-df648330fc4e
2026-03-30 12:12:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252Fe8d8531dba4b4aa5b534609ca290a230.jpg%27 HTTP/1.1" 200 None
2026-03-30 12:12:01 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/e8d8531dba4b4aa5b534609ca290a230.jpg not downloaded yet
2026-03-30 12:12:01 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-30 12:12:01 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: 648059b1-008d-52fc-b06c-fc73234dbdc8 with identifier: 202601179216399
2026-03-30 12:12:01 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 648059b1-008d-52fc-b06c-fc73234dbdc8
2026-03-30 12:12:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F859aa26a8a37481dab1dc5984f1f1129.jpg%27 HTTP/1.1" 200 None
2026-03-30 12:12:01 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/859aa26a8a37481dab1dc5984f1f1129.jpg not downloaded yet
2026-03-30 12:12:01 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-30 12:12:01 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: 1227c7e1-d8a0-5589-b783-40caf0d6b018 with identifier: 202601159154661
2026-03-30 12:12:01 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 1227c7e1-d8a0-5589-b783-40caf0d6b018
2026-03-30 12:12:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F98235c0cac6344fd85f2db7fe3fe2db9.jpg%27 HTTP/1.1" 200 None
2026-03-30 12:12:01 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/98235c0cac6344fd85f2db7fe3fe2db9.jpg not downloaded yet
2026-03-30 12:12:01 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/98235c0cac6344fd85f2db7fe3fe2db9.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-30 12:12:01 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/b48768ccd35043ebba6fcc56b6a4665b.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-30 12:12:01 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/859aa26a8a37481dab1dc5984f1f1129.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-30 12:12:01 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/77cc393b9a1842a8aab2c8b83b60ec23.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-30 12:12:01 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/e8d8531dba4b4aa5b534609ca290a230.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-30 12:12:01 [common.azure_blobstore] INFO: Using managed identity for Azure Blob Storage
2026-03-30 12:12:01 [common.azure_blobstore] INFO: Using managed identity with client ID: 39192755-105d-4ce2-a21d-ab53b63f8b87
2026-03-30 12:12:01 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-30 12:12:01 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fc589b5d710>
2026-03-30 12:12:01 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-30 12:12:01 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2026-03-30 12:12:01 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 2111
2026-03-30 12:12:01 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxproduction.blob.core.windows.net:443
2026-03-30 12:12:01 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-30 12:12:01 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-30 12:12:01 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_1227c7e1-d8a0-5589-b783-40caf0d6b018/1227c7e1-d8a0-5589-b783-40caf0d6b018_image_0.jpg HTTP/1.1" 201 0
2026-03-30 12:12:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:01 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F98235c0cac6344fd85f2db7fe3fe2db9.jpg') HTTP/1.1" 204 0
2026-03-30 12:12:01 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/98235c0cac6344fd85f2db7fe3fe2db9.jpg saved to cache
2026-03-30 12:12:01 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 1227c7e1-d8a0-5589-b783-40caf0d6b018
2026-03-30 12:12:01 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 1227c7e1-d8a0-5589-b783-40caf0d6b018 sending to next pipeline
2026-03-30 12:12:01 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: 1227c7e1-d8a0-5589-b783-40caf0d6b018 with identifier 202601159154661
2026-03-30 12:12:01 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: None -> <ConnectionState.START: 0>
2026-03-30 12:12:01 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.START: 0> -> <ConnectionState.HDR_SENT: 2>
2026-03-30 12:12:01 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.HDR_SENT: 2> -> <ConnectionState.HDR_SENT: 2>
2026-03-30 12:12:01 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.HDR_SENT: 2> -> <ConnectionState.OPEN_PIPE: 4>
2026-03-30 12:12:01 [azure.servicebus._pyamqp.session] INFO: Session state changed: <SessionState.UNMAPPED: 0> -> <SessionState.BEGIN_SENT: 1>
2026-03-30 12:12:01 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACHED: 0> -> <LinkState.ATTACH_SENT: 1>
2026-03-30 12:12:01 [azure.servicebus._pyamqp.management_link] INFO: Management link receiver state changed: <LinkState.DETACHED: 0> -> <LinkState.ATTACH_SENT: 1>
2026-03-30 12:12:01 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACHED: 0> -> <LinkState.ATTACH_SENT: 1>
2026-03-30 12:12:01 [azure.servicebus._pyamqp.management_link] INFO: Management link sender state changed: <LinkState.DETACHED: 0> -> <LinkState.ATTACH_SENT: 1>
2026-03-30 12:12:01 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.OPEN_PIPE: 4> -> <ConnectionState.OPEN_SENT: 7>
2026-03-30 12:12:01 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.OPEN_SENT: 7> -> <ConnectionState.OPENED: 9>
2026-03-30 12:12:01 [azure.servicebus._pyamqp.session] INFO: Session state changed: <SessionState.BEGIN_SENT: 1> -> <SessionState.MAPPED: 3>
2026-03-30 12:12:01 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACH_SENT: 1> -> <LinkState.ATTACHED: 3>
2026-03-30 12:12:01 [azure.servicebus._pyamqp.management_link] INFO: Management link receiver state changed: <LinkState.ATTACH_SENT: 1> -> <LinkState.ATTACHED: 3>
2026-03-30 12:12:02 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACH_SENT: 1> -> <LinkState.ATTACHED: 3>
2026-03-30 12:12:02 [azure.servicebus._pyamqp.management_link] INFO: Management link sender state changed: <LinkState.ATTACH_SENT: 1> -> <LinkState.ATTACHED: 3>
2026-03-30 12:12:02 [azure.servicebus._pyamqp.cbs] DEBUG: CBS completed opening with status: <ManagementOpenResult.OK: 1>
2026-03-30 12:12:02 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2026-03-30 12:12:02 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 2026
2026-03-30 12:12:02 [azure.identity._credentials.chained] INFO: DefaultAzureCredential acquired a token from WorkloadIdentityCredential
2026-03-30 12:12:02 [azure.servicebus._pyamqp.cbs] DEBUG: CBS update in progress. Token put time: 1774872722
2026-03-30 12:12:02 [azure.servicebus._pyamqp.cbs] DEBUG: CBS update in progress. Token put time: 1774872722
2026-03-30 12:12:02 [azure.servicebus._pyamqp.cbs] DEBUG: CBS Put token result (<ManagementExecuteOperationResult.OK: 0>), status code: 202, status_description: b'Accepted'.
2026-03-30 12:12:02 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:02 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACHED: 0> -> <LinkState.ATTACH_SENT: 1>
2026-03-30 12:12:02 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACH_SENT: 1> -> <LinkState.ATTACHED: 3>
2026-03-30 12:12:02 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:02 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:02 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:02 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:02 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item 1227c7e1-d8a0-5589-b783-40caf0d6b018
2026-03-30 12:12:02 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/98235c0cac6344fd85f2db7fe3fe2db9.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 42,
 'color': 'White',
 'currency': 'GBP',
 'description': '1 OWNER FROM NEW. EXCEPTIONAL CONDITION BOTH INSIDE AND OUT '
                'WITH A FULL VOLVO MAIN DEALER SERVICE HISTORY. LAST AT 58K. '
                'GREAT SPEC R DESIGN. DRIVES WITHOUT FAULT. SUPPLIED WITH A '
                "NEW MOT. WARRANTY. 2 KEY'S. PART EXCHNAGE WELCOME. FINANCE "
                'ARRANGED. JUST ARRIVED MORE PICTIRES TO FOLLOW.\n'
                ' 2021 Volvo V60 T6 R-Design is a superb example, boasting a '
                'full dealership service history and having had only one owner '
                'from new. This plug-in hybrid estate features a powerful 2.0 '
                'litre petrol engine paired with Four Wheel Drive, ensuring '
                'excellent performance and all-weather capability. It meets '
                'Euro 6 emissions standards and comes with a 3 month warranty '
                'for your peace of mind, alongside a 12 month MOT provided on '
                'purchase. Notable features include seamless Smartphone '
                'Integration with Apple CarPlay and Android Auto, a 7m Type 2 '
                'Mode 3 Charge Cable, striking 19in Diamond Cut Matt Black '
                'alloy wheels, and a 2-Zone Electronic Climate Control system '
                'with CleanZone Air Quality. Keyless Drive with Keyless Entry '
                'adds to the convenience.\n'
                '\n'
                'The V60 R-Design offers a compelling blend of efficiency and '
                'exhilarating performance, accelerating from 0-60mph in a '
                'swift 5.2 seconds, a figure that sets it apart from many '
                'other vehicles in its class. Its generous boot space makes it '
                'incredibly practical for everyday life and longer journeys '
                'alike. The inclusion of features like the advanced climate '
                'control system with a pollen filter and the ease of keyless '
                'operation further enhance the driving experience, '
                'demonstrating a commitment to both comfort and cutting edge '
                'technology.',
 'doors_number': 5,
 'emission_standard': 'Euro 6',
 'engine': '2L',
 'engine_horse_power': 335,
 'fuel_type': 'Petrol Plug-in Hybrid',
 'id': '1227c7e1-d8a0-5589-b783-40caf0d6b018',
 'is_damaged': False,
 'is_operable': True,
 'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202601159154661',
 'listing': 'directbuy',
 'location': 'Portsmouth, Hampshire',
 'make': 'Volvo',
 'model': 'V60',
 'odometer': 62000,
 'odometer_reading_unit': 'mi',
 'origin_country_code': 'GB',
 'original_photo_count': 32,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_1227c7e1-d8a0-5589-b783-40caf0d6b018/1227c7e1-d8a0-5589-b783-40caf0d6b018_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 398210,
             'Height': 1152,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/98235c0cac6344fd85f2db7fe3fe2db9.jpg',
             'Width': 2048}],
 'price': 19995,
 'price_includes_vat': True,
 'registration_date': '2021-12-16',
 'seats_number': 5,
 'seller_name': None,
 'title': 'Volvo V60 2.0h T6 Recharge 11.6kWh R-Design Auto AWD Euro 6 (s/s) '
          '5dr',
 'transmission': 'automatic',
 'trim': 'R-Design',
 'year': 2021}
2026-03-30 12:12:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202601159154661%27 HTTP/1.1" 200 None
2026-03-30 12:12:02 [auto_selling_autotrader.co.uk] INFO: Saving data for 202601159154661: {'created_time': 1774872722.479701, 'last_price_update_time': 1774872722.479713}
2026-03-30 12:12:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202601159154661') HTTP/1.1" 204 0
2026-03-30 12:12:02 [scrapy.core.engine] DEBUG: Crawled (200) <POST https://www.autotrader.co.uk/at-graphql> (referer: https://www.autotrader.co.uk/at-gateway?opname=SearchResultsListingsGridQuery&opname=SearchResultsFacetsWithGroupsQuery) ['zyte-api']
2026-03-30 12:12:02 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-30 12:12:02 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fc5897949d0>
2026-03-30 12:12:02 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-30 12:12:02 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-30 12:12:02 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-30 12:12:02 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_3c8f2bee-3be6-500b-b537-3e2e2bb27b10/3c8f2bee-3be6-500b-b537-3e2e2bb27b10_image_0.jpg HTTP/1.1" 201 0
2026-03-30 12:12:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252Fb48768ccd35043ebba6fcc56b6a4665b.jpg') HTTP/1.1" 204 0
2026-03-30 12:12:02 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/b48768ccd35043ebba6fcc56b6a4665b.jpg saved to cache
2026-03-30 12:12:02 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 3c8f2bee-3be6-500b-b537-3e2e2bb27b10
2026-03-30 12:12:02 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 3c8f2bee-3be6-500b-b537-3e2e2bb27b10 sending to next pipeline
2026-03-30 12:12:02 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-30 12:12:02 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fc5883a6310>
2026-03-30 12:12:02 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-30 12:12:02 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-30 12:12:02 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-30 12:12:02 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_648059b1-008d-52fc-b06c-fc73234dbdc8/648059b1-008d-52fc-b06c-fc73234dbdc8_image_0.jpg HTTP/1.1" 201 0
2026-03-30 12:12:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:02 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F859aa26a8a37481dab1dc5984f1f1129.jpg') HTTP/1.1" 204 0
2026-03-30 12:12:02 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/859aa26a8a37481dab1dc5984f1f1129.jpg saved to cache
2026-03-30 12:12:02 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 648059b1-008d-52fc-b06c-fc73234dbdc8
2026-03-30 12:12:02 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 648059b1-008d-52fc-b06c-fc73234dbdc8 sending to next pipeline
2026-03-30 12:12:02 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-30 12:12:02 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fc58b2f34d0>
2026-03-30 12:12:02 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-30 12:12:02 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-30 12:12:02 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_f5eaaaa7-6b74-598c-9001-7f99e69425c2/f5eaaaa7-6b74-598c-9001-7f99e69425c2_image_0.jpg HTTP/1.1" 201 0
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F77cc393b9a1842a8aab2c8b83b60ec23.jpg') HTTP/1.1" 204 0
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/77cc393b9a1842a8aab2c8b83b60ec23.jpg saved to cache
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item f5eaaaa7-6b74-598c-9001-7f99e69425c2
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item f5eaaaa7-6b74-598c-9001-7f99e69425c2 sending to next pipeline
2026-03-30 12:12:03 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-30 12:12:03 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fc589750910>
2026-03-30 12:12:03 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-30 12:12:03 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_a509fd1a-780f-5f24-9ea2-df648330fc4e/a509fd1a-780f-5f24-9ea2-df648330fc4e_image_0.jpg HTTP/1.1" 201 0
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252Fe8d8531dba4b4aa5b534609ca290a230.jpg') HTTP/1.1" 204 0
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/e8d8531dba4b4aa5b534609ca290a230.jpg saved to cache
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item a509fd1a-780f-5f24-9ea2-df648330fc4e
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item a509fd1a-780f-5f24-9ea2-df648330fc4e sending to next pipeline
2026-03-30 12:12:03 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: 3c8f2bee-3be6-500b-b537-3e2e2bb27b10 with identifier 202602169955425
2026-03-30 12:12:03 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:03 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:03 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item 3c8f2bee-3be6-500b-b537-3e2e2bb27b10
2026-03-30 12:12:03 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/b48768ccd35043ebba6fcc56b6a4665b.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 116,
 'color': 'White',
 'currency': 'GBP',
 'description': 'WELL LOOKED AFTER ,WITH FULL MAIN DEALER SERVICE HISTORY '
                '.UPGRATED MEDIA SOUND SYSYTEM . SUPPLIED WITH 2 KEYS NEW MOT '
                'AND WARRANTY .2016 Suzuki Swift SZ-L with exceptionally low '
                'mileage for its year, having covered just 41,000 miles. This '
                'SZ-L model is powered by an efficient 1.2 litre petrol engine '
                'and meets Euro 6 emissions standards. It comes equipped with '
                'integrated Bluetooth for seamless audio streaming and '
                'handsfree calls, alongside air conditioning to ensure a '
                'comfortable cabin environment.\n'
                '\n'
                'This Suzuki Swift SZ-L stands out with its very low mileage, '
                'indicating it has been used sparingly and is in excellent '
                'condition for its age. Furthermore, its placement in a low '
                'insurance group makes it an economical choice for many '
                'drivers, suggesting lower running costs compared to similar '
                'vehicles. This combination of low mileage, efficient engine, '
                'and affordable insurance makes this Swift a sensible and '
                'appealing option.',
 'doors_number': 5,
 'emission_standard': 'Euro 6',
 'engine': '1.2L',
 'engine_horse_power': 93,
 'fuel_type': 'Petrol',
 'id': '3c8f2bee-3be6-500b-b537-3e2e2bb27b10',
 'is_damaged': False,
 'is_operable': True,
 'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202602169955425',
 'listing': 'directbuy',
 'location': 'Portsmouth, Hampshire',
 'make': 'Suzuki',
 'model': 'Swift',
 'odometer': 41000,
 'odometer_reading_unit': 'mi',
 'origin_country_code': 'GB',
 'original_photo_count': 20,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_3c8f2bee-3be6-500b-b537-3e2e2bb27b10/3c8f2bee-3be6-500b-b537-3e2e2bb27b10_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 360415,
             'Height': 1152,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/b48768ccd35043ebba6fcc56b6a4665b.jpg',
             'Width': 2048}],
 'price': 7150,
 'price_includes_vat': True,
 'registration_date': '2016-06-04',
 'seats_number': 5,
 'seller_name': None,
 'title': 'Suzuki Swift 1.2 SZ-L Euro 6 5dr',
 'transmission': 'manual',
 'trim': 'SZ-L',
 'year': 2016}
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602169955425%27 HTTP/1.1" 200 None
2026-03-30 12:12:03 [auto_selling_autotrader.co.uk] INFO: Saving data for 202602169955425: {'created_time': 1774872723.272856, 'last_price_update_time': 1774872723.272869}
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202602169955425') HTTP/1.1" 204 0
2026-03-30 12:12:03 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: 648059b1-008d-52fc-b06c-fc73234dbdc8 with identifier 202601179216399
2026-03-30 12:12:03 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:03 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:03 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item 648059b1-008d-52fc-b06c-fc73234dbdc8
2026-03-30 12:12:03 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/859aa26a8a37481dab1dc5984f1f1129.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 102,
 'color': 'Red',
 'currency': 'GBP',
 'description': 'EXCELLENT CONDITION FOR AGE AND MILEAGE , WITH FULL SERVICE '
                'HISTORY . JUST HAD A NEW CAMBELT AND WATER PUMP This 2017 '
                'Volkswagen Golf TDI Match Edition presents an exceptional '
                'opportunity with its remarkably low mileage of just 48,400 '
                'miles. This Euro 6 compliant 1.6 litre diesel model, in its '
                'popular Match Edition trim, offers a refined driving '
                'experience. Inside, you will find comfort features such as '
                'manual air conditioning and heated front seats, ensuring a '
                'pleasant journey in any weather. Practicality is also a key '
                'consideration, with parking sensors fitted at both the front '
                'and rear to aid manoeuvring, and automatic dusk sensing '
                'headlights for enhanced visibility. Connectivity is made '
                'simple with Bluetooth telephone preparation, allowing you to '
                'stay in touch on the go.\n'
                '\n'
                'The benefits of choosing this particular Golf are clear when '
                'looking at its highlights. Achieving an impressive 72 miles '
                'per gallon, it offers outstanding fuel economy, making it an '
                'economical choice for your motoring needs. Furthermore, its '
                'low CO2 emissions contribute to a more environmentally '
                'friendly drive. The generous boot space provides ample room '
                'for luggage or shopping, adding to its everyday usability. '
                'This vehicle also sits within a low insurance group, which '
                'can help to keep your running costs down, making it a '
                'sensible and appealing option compared to many other vehicles '
                'in its class.',
 'doors_number': 5,
 'emission_standard': 'Euro 6',
 'engine': '1.6L',
 'engine_horse_power': 108,
 'fuel_type': 'Diesel',
 'id': '648059b1-008d-52fc-b06c-fc73234dbdc8',
 'is_damaged': False,
 'is_operable': True,
 'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202601179216399',
 'listing': 'directbuy',
 'location': 'Portsmouth, Hampshire',
 'make': 'Volkswagen',
 'model': 'Golf',
 'odometer': 48400,
 'odometer_reading_unit': 'mi',
 'origin_country_code': 'GB',
 'original_photo_count': 19,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_648059b1-008d-52fc-b06c-fc73234dbdc8/648059b1-008d-52fc-b06c-fc73234dbdc8_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 316403,
             'Height': 1152,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/859aa26a8a37481dab1dc5984f1f1129.jpg',
             'Width': 2048}],
 'price': 10995,
 'price_includes_vat': True,
 'registration_date': '2017-03-08',
 'seats_number': 5,
 'seller_name': None,
 'title': 'Volkswagen Golf 1.6 TDI BlueMotion Tech Match Edition DSG Euro 6 '
          '(s/s) 5dr',
 'transmission': 'automatic',
 'trim': 'Match Edition',
 'year': 2017}
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202601179216399%27 HTTP/1.1" 200 None
2026-03-30 12:12:03 [auto_selling_autotrader.co.uk] INFO: Saving data for 202601179216399: {'created_time': 1774872723.336144, 'last_price_update_time': 1774872723.336156}
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202601179216399') HTTP/1.1" 204 0
2026-03-30 12:12:03 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: f5eaaaa7-6b74-598c-9001-7f99e69425c2 with identifier 202602129872004
2026-03-30 12:12:03 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:03 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:03 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item f5eaaaa7-6b74-598c-9001-7f99e69425c2
2026-03-30 12:12:03 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/77cc393b9a1842a8aab2c8b83b60ec23.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 121,
 'color': 'Black',
 'currency': 'GBP',
 'description': 'EXCEPTIONAL CONDITION BOTH INSIDE AND OUT WITH A FULL SERVICE '
                'HISTORY. LEXUS WARRANTY UNTIL 12/26. PREMIUM EDITION WITH '
                'SUNROOF. SATNAV. HEATED LEATHER. DRIVES WITH OUT FAULT. '
                'ADVISORY FREE MOT. PART EWXCHANGE WELCOME. FINANCE ARRANGED.',
 'doors_number': 4,
 'emission_standard': 'Euro 6',
 'engine': '2.5L',
 'engine_horse_power': 215,
 'fuel_type': 'Petrol Hybrid',
 'id': 'f5eaaaa7-6b74-598c-9001-7f99e69425c2',
 'is_damaged': False,
 'is_operable': True,
 'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202602129872004',
 'listing': 'directbuy',
 'location': 'Portsmouth, Hampshire',
 'make': 'Lexus',
 'model': 'ES',
 'odometer': 28000,
 'odometer_reading_unit': 'mi',
 'origin_country_code': 'GB',
 'original_photo_count': 26,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_f5eaaaa7-6b74-598c-9001-7f99e69425c2/f5eaaaa7-6b74-598c-9001-7f99e69425c2_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 606457,
             'Height': 1536,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/77cc393b9a1842a8aab2c8b83b60ec23.jpg',
             'Width': 2048}],
 'price': 25995,
 'price_includes_vat': True,
 'registration_date': '2022-12-31',
 'seats_number': 5,
 'seller_name': None,
 'title': 'Lexus ES 2.5 300h Premium Edition Saloon 4dr Petrol Hybrid E-CVT '
          'Euro 6 (s/s) (218 ps)',
 'transmission': 'automatic',
 'trim': 'Premium Edition',
 'year': 2022}
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602129872004%27 HTTP/1.1" 200 None
2026-03-30 12:12:03 [auto_selling_autotrader.co.uk] INFO: Saving data for 202602129872004: {'created_time': 1774872723.388982, 'last_price_update_time': 1774872723.388993}
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202602129872004') HTTP/1.1" 204 0
2026-03-30 12:12:03 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: a509fd1a-780f-5f24-9ea2-df648330fc4e with identifier 202601259405817
2026-03-30 12:12:03 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:03 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:03 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item a509fd1a-780f-5f24-9ea2-df648330fc4e
2026-03-30 12:12:03 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/e8d8531dba4b4aa5b534609ca290a230.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 185,
 'color': 'Black',
 'currency': 'GBP',
 'description': 'WELL LOOKED AFTER , WITH FULL SERVICE HISTORY . JUST HAD BOTH '
                "CAM BELTS REPLACED . SUPPLIED WITH 2 KEY'S NEW MOT AND "
                'WARRANTY .This 2017 Land Rover Range Rover Sport SD V6 HSE '
                'Dynamic boasts a full service history right up to its last '
                'service at 85,000 miles, ensuring excellent provenance. This '
                'capable four wheel drive vehicle is powered by a 3.0L Euro 6 '
                'diesel engine and offers a substantial towing weight of 3500 '
                "kg. Inside, you'll find luxurious heated front and rear "
                'seats, 14 way powered front seats with memory function, and '
                'convenient keyless entry. For added ease, it includes a rear '
                'view camera and Bluetooth phone connectivity, all presented '
                'with a clear vehicle history check and a 6 month warranty.\n'
                '\n'
                'Experience impressive performance with this Range Rover '
                'Sport, capable of accelerating from 0 to 60mph in just 6.8 '
                'seconds, a testament to its dynamic capabilities. The HSE '
                'Dynamic trim level signifies a higher specification, offering '
                'a superior driving experience compared to other models. '
                "You'll also appreciate the generous boot space, perfect for "
                'all your luggage or equipment, and the reassurance of a 12 '
                'month MOT provided on purchase.',
 'doors_number': 5,
 'emission_standard': 'Euro 6',
 'engine': '3L',
 'engine_horse_power': 302,
 'fuel_type': 'Diesel',
 'id': 'a509fd1a-780f-5f24-9ea2-df648330fc4e',
 'is_damaged': False,
 'is_operable': True,
 'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202601259405817',
 'listing': 'directbuy',
 'location': 'Portsmouth, Hampshire',
 'make': 'Land Rover',
 'model': 'Range Rover Sport',
 'odometer': 87000,
 'odometer_reading_unit': 'mi',
 'origin_country_code': 'GB',
 'original_photo_count': 17,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_a509fd1a-780f-5f24-9ea2-df648330fc4e/a509fd1a-780f-5f24-9ea2-df648330fc4e_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 442716,
             'Height': 1152,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/e8d8531dba4b4aa5b534609ca290a230.jpg',
             'Width': 2048}],
 'price': 16995,
 'price_includes_vat': True,
 'registration_date': '2017-03-17',
 'seats_number': 5,
 'seller_name': None,
 'title': 'Land Rover Range Rover Sport 3.0 SD V6 HSE Dynamic Auto 4WD Euro 6 '
          '(s/s) 5dr',
 'transmission': 'automatic',
 'trim': 'HSE Dynamic',
 'year': 2017}
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202601259405817%27 HTTP/1.1" 200 None
2026-03-30 12:12:03 [auto_selling_autotrader.co.uk] INFO: Saving data for 202601259405817: {'created_time': 1774872723.435809, 'last_price_update_time': 1774872723.43582}
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202601259405817') HTTP/1.1" 204 0
2026-03-30 12:12:03 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-30 12:12:03 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: 4c9168d4-88b7-5d50-9ea9-45d7778105a0 with identifier: 202603180791647
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 4c9168d4-88b7-5d50-9ea9-45d7778105a0
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F7b72e897ed864499996e7fe82315f58f.jpg%27 HTTP/1.1" 200 None
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/7b72e897ed864499996e7fe82315f58f.jpg not downloaded yet
2026-03-30 12:12:03 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-30 12:12:03 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: 4b46a5e4-ef67-5d7c-9dc7-2401d82b24c7 with identifier: 202603160740452
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 4b46a5e4-ef67-5d7c-9dc7-2401d82b24c7
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F97881db3b3d84214ac26c3f4f288b4d6.jpg%27 HTTP/1.1" 200 None
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/97881db3b3d84214ac26c3f4f288b4d6.jpg not downloaded yet
2026-03-30 12:12:03 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-30 12:12:03 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: 1de69e53-e053-5042-b850-448af59d2b4c with identifier: 202603040405261
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 1de69e53-e053-5042-b850-448af59d2b4c
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F32254f4d86e7433d909cf2ebfaed0e21.jpg%27 HTTP/1.1" 200 None
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/32254f4d86e7433d909cf2ebfaed0e21.jpg not downloaded yet
2026-03-30 12:12:03 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-30 12:12:03 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: 04d9c655-ca60-5315-9d3c-0ecf8caca08a with identifier: 202603040386452
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 04d9c655-ca60-5315-9d3c-0ecf8caca08a
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F758db978b19a46b79d65546735ce119d.jpg%27 HTTP/1.1" 200 None
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/758db978b19a46b79d65546735ce119d.jpg not downloaded yet
2026-03-30 12:12:03 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-30 12:12:03 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: 4f38df39-7ea5-512a-b214-e6b2d8f2c392 with identifier: 202603020320093
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 4f38df39-7ea5-512a-b214-e6b2d8f2c392
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F5df61fd462ba43729477492fb21a6877.jpg%27 HTTP/1.1" 200 None
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/5df61fd462ba43729477492fb21a6877.jpg not downloaded yet
2026-03-30 12:12:03 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-30 12:12:03 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: 8d5ed4f4-e788-5732-951f-055a28676468 with identifier: 202602250206482
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 8d5ed4f4-e788-5732-951f-055a28676468
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252Fbcae73711fd24ae9842baaf6451335d1.jpg%27 HTTP/1.1" 200 None
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/bcae73711fd24ae9842baaf6451335d1.jpg not downloaded yet
2026-03-30 12:12:03 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-30 12:12:03 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: 46c48132-41d5-51d7-8155-26e97a8e0b8a with identifier: 202602200085700
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 46c48132-41d5-51d7-8155-26e97a8e0b8a
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252Feb43a9b1d06a4966b129f667019b65ff.jpg%27 HTTP/1.1" 200 None
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/eb43a9b1d06a4966b129f667019b65ff.jpg not downloaded yet
2026-03-30 12:12:03 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-30 12:12:03 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: dc2156ba-529d-59be-b5c5-fb93cbff1885 with identifier: 202602200074354
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item dc2156ba-529d-59be-b5c5-fb93cbff1885
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F0c06b652c4484bd2933589c671a07f0e.jpg%27 HTTP/1.1" 200 None
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/0c06b652c4484bd2933589c671a07f0e.jpg not downloaded yet
2026-03-30 12:12:03 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-30 12:12:03 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: 284d050d-a9f7-5090-8227-72ddbb05b609 with identifier: 202602190052151
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 284d050d-a9f7-5090-8227-72ddbb05b609
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F4313e7b094984c218b703fb792d415a6.jpg%27 HTTP/1.1" 200 None
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/4313e7b094984c218b703fb792d415a6.jpg not downloaded yet
2026-03-30 12:12:03 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/7b72e897ed864499996e7fe82315f58f.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-30 12:12:03 [auto_selling_autotrader.co.uk] INFO: Extracted item from batch: None
2026-03-30 12:12:03 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: c9db9e88-d61b-54b2-8dcf-3f0e6372411d with identifier: 202602169963132
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item c9db9e88-d61b-54b2-8dcf-3f0e6372411d
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F73a90cd04b1d49119443d99eb8b0fc0b.jpg%27 HTTP/1.1" 200 None
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/73a90cd04b1d49119443d99eb8b0fc0b.jpg not downloaded yet
2026-03-30 12:12:03 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/97881db3b3d84214ac26c3f4f288b4d6.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-30 12:12:03 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/32254f4d86e7433d909cf2ebfaed0e21.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-30 12:12:03 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/758db978b19a46b79d65546735ce119d.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-30 12:12:03 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/4313e7b094984c218b703fb792d415a6.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-30 12:12:03 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-30 12:12:03 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fc588361350>
2026-03-30 12:12:03 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-30 12:12:03 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_4c9168d4-88b7-5d50-9ea9-45d7778105a0/4c9168d4-88b7-5d50-9ea9-45d7778105a0_image_0.jpg HTTP/1.1" 201 0
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F7b72e897ed864499996e7fe82315f58f.jpg') HTTP/1.1" 204 0
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/7b72e897ed864499996e7fe82315f58f.jpg saved to cache
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 4c9168d4-88b7-5d50-9ea9-45d7778105a0
2026-03-30 12:12:03 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 4c9168d4-88b7-5d50-9ea9-45d7778105a0 sending to next pipeline
2026-03-30 12:12:03 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: 4c9168d4-88b7-5d50-9ea9-45d7778105a0 with identifier 202603180791647
2026-03-30 12:12:03 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:03 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:03 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item 4c9168d4-88b7-5d50-9ea9-45d7778105a0
2026-03-30 12:12:03 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/7b72e897ed864499996e7fe82315f58f.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 175,
 'color': 'White',
 'currency': 'GBP',
 'description': 'This 2014 Kia Sorento CRDi KX-2 presents an exceptional '
                'opportunity with its remarkably low mileage of just 63,800 '
                'miles, especially for a vehicle of its age, and benefits from '
                'a full service history. This Four Wheel Drive model, powered '
                'by a robust 2.2L Diesel engine, offers comfortable seating '
                'for seven occupants. It comes with a clear vehicle history '
                'check and a 6 months warranty for your peace of mind, '
                'alongside a 12 months MOT provided on purchase. Notable '
                'features include Bluetooth with Voice Recognition and Music '
                'Streaming, a Reversing Camera System with parking assistance, '
                'Automatic Headlight Control, and Heated Front Seats, all '
                'contributing to a refined driving experience.\n'
                '\n'
                'The Sorento KX-2 stands out due to its very low mileage for '
                'its age, indicating less wear and tear compared to other '
                'similar vehicles on the market. Its full service history and '
                'the fact that it has undergone a clear vehicle history check '
                'further enhance its appeal, suggesting a well maintained and '
                'reliable vehicle. The inclusion of a 6 months warranty and a '
                'fresh 12 months MOT on purchase provides added confidence for '
                'the next owner. Furthermore, its position within a low '
                'insurance group makes it an economical choice for many '
                'drivers, combining practicality with desirable features.',
 'doors_number': 5,
 'emission_standard': 'Euro 5',
 'engine': '2.2L',
 'engine_horse_power': 194,
 'fuel_type': 'Diesel',
 'id': '4c9168d4-88b7-5d50-9ea9-45d7778105a0',
 'is_damaged': False,
 'is_operable': True,
 'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202603180791647',
 'listing': 'directbuy',
 'location': 'Portsmouth, Hampshire',
 'make': 'Kia',
 'model': 'Sorento',
 'odometer': 63800,
 'odometer_reading_unit': 'mi',
 'origin_country_code': 'GB',
 'original_photo_count': 17,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_4c9168d4-88b7-5d50-9ea9-45d7778105a0/4c9168d4-88b7-5d50-9ea9-45d7778105a0_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 334406,
             'Height': 1152,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/7b72e897ed864499996e7fe82315f58f.jpg',
             'Width': 2048}],
 'price': 9495,
 'price_includes_vat': True,
 'registration_date': '2014-12-31',
 'seats_number': 7,
 'seller_name': None,
 'title': 'KIA Sorento 2.2 CRDi KX-2 Auto AWD Euro 5 5dr',
 'transmission': 'automatic',
 'trim': 'KX-2',
 'year': 2014}
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603180791647%27 HTTP/1.1" 200 None
2026-03-30 12:12:03 [auto_selling_autotrader.co.uk] INFO: Saving data for 202603180791647: {'created_time': 1774872723.976817, 'last_price_update_time': 1774872723.976829}
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:03 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202603180791647') HTTP/1.1" 204 0
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fc588394750>
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_4b46a5e4-ef67-5d7c-9dc7-2401d82b24c7/4b46a5e4-ef67-5d7c-9dc7-2401d82b24c7_image_0.jpg HTTP/1.1" 201 0
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F97881db3b3d84214ac26c3f4f288b4d6.jpg') HTTP/1.1" 204 0
2026-03-30 12:12:04 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/97881db3b3d84214ac26c3f4f288b4d6.jpg saved to cache
2026-03-30 12:12:04 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 4b46a5e4-ef67-5d7c-9dc7-2401d82b24c7
2026-03-30 12:12:04 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 4b46a5e4-ef67-5d7c-9dc7-2401d82b24c7 sending to next pipeline
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fc58832b2d0>
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_1de69e53-e053-5042-b850-448af59d2b4c/1de69e53-e053-5042-b850-448af59d2b4c_image_0.jpg HTTP/1.1" 201 0
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F32254f4d86e7433d909cf2ebfaed0e21.jpg') HTTP/1.1" 204 0
2026-03-30 12:12:04 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/32254f4d86e7433d909cf2ebfaed0e21.jpg saved to cache
2026-03-30 12:12:04 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 1de69e53-e053-5042-b850-448af59d2b4c
2026-03-30 12:12:04 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 1de69e53-e053-5042-b850-448af59d2b4c sending to next pipeline
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fc5883d8190>
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_04d9c655-ca60-5315-9d3c-0ecf8caca08a/04d9c655-ca60-5315-9d3c-0ecf8caca08a_image_0.jpg HTTP/1.1" 201 0
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F758db978b19a46b79d65546735ce119d.jpg') HTTP/1.1" 204 0
2026-03-30 12:12:04 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/758db978b19a46b79d65546735ce119d.jpg saved to cache
2026-03-30 12:12:04 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 04d9c655-ca60-5315-9d3c-0ecf8caca08a
2026-03-30 12:12:04 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 04d9c655-ca60-5315-9d3c-0ecf8caca08a sending to next pipeline
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fc5883d9d10>
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_284d050d-a9f7-5090-8227-72ddbb05b609/284d050d-a9f7-5090-8227-72ddbb05b609_image_0.jpg HTTP/1.1" 201 0
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F4313e7b094984c218b703fb792d415a6.jpg') HTTP/1.1" 204 0
2026-03-30 12:12:04 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/4313e7b094984c218b703fb792d415a6.jpg saved to cache
2026-03-30 12:12:04 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 284d050d-a9f7-5090-8227-72ddbb05b609
2026-03-30 12:12:04 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 284d050d-a9f7-5090-8227-72ddbb05b609 sending to next pipeline
2026-03-30 12:12:04 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/73a90cd04b1d49119443d99eb8b0fc0b.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-30 12:12:04 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: 4b46a5e4-ef67-5d7c-9dc7-2401d82b24c7 with identifier 202603160740452
2026-03-30 12:12:04 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:04 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:04 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item 4b46a5e4-ef67-5d7c-9dc7-2401d82b24c7
2026-03-30 12:12:04 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/97881db3b3d84214ac26c3f4f288b4d6.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 122,
 'color': 'Silver',
 'currency': 'GBP',
 'description': 'Discover this exceptional 2016 Ford Fiesta Zetec, boasting '
                'remarkably low mileage for its age, having covered just '
                '36,000 miles. This Euro 6 compliant 1.25 litre petrol model '
                'comes with a full service history and has had only two '
                'owners. A clear vehicle history check provides peace of mind, '
                'and it will be supplied with 12 months MOT and a 3 month '
                'warranty for your assurance.\n'
                '\n'
                'This Fiesta Zetec stands out with its Bluetooth hands free '
                'phone system, allowing for convenient connectivity on the go. '
                'Its low insurance group rating is a significant advantage, '
                'indicating lower running costs compared to many other '
                'vehicles. With its comprehensive service record and low '
                'mileage, this Ford Fiesta represents a well maintained and '
                'economical choice.',
 'doors_number': 5,
 'emission_standard': 'Euro 6',
 'engine': '1.25L',
 'engine_horse_power': 81,
 'fuel_type': 'Petrol',
 'id': '4b46a5e4-ef67-5d7c-9dc7-2401d82b24c7',
 'is_damaged': False,
 'is_operable': True,
 'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202603160740452',
 'listing': 'directbuy',
 'location': 'Portsmouth, Hampshire',
 'make': 'Ford',
 'model': 'Fiesta',
 'odometer': 36000,
 'odometer_reading_unit': 'mi',
 'origin_country_code': 'GB',
 'original_photo_count': 13,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_4b46a5e4-ef67-5d7c-9dc7-2401d82b24c7/4b46a5e4-ef67-5d7c-9dc7-2401d82b24c7_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 339767,
             'Height': 1152,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/97881db3b3d84214ac26c3f4f288b4d6.jpg',
             'Width': 2048}],
 'price': 6995,
 'price_includes_vat': True,
 'registration_date': '2016-07-27',
 'seats_number': 5,
 'seller_name': None,
 'title': 'Ford Fiesta 1.25 Zetec Euro 6 5dr',
 'transmission': 'manual',
 'trim': 'Zetec',
 'year': 2016}
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603160740452%27 HTTP/1.1" 200 None
2026-03-30 12:12:04 [auto_selling_autotrader.co.uk] INFO: Saving data for 202603160740452: {'created_time': 1774872724.444285, 'last_price_update_time': 1774872724.444296}
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202603160740452') HTTP/1.1" 204 0
2026-03-30 12:12:04 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: 1de69e53-e053-5042-b850-448af59d2b4c with identifier 202603040405261
2026-03-30 12:12:04 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:04 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:04 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item 1de69e53-e053-5042-b850-448af59d2b4c
2026-03-30 12:12:04 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/32254f4d86e7433d909cf2ebfaed0e21.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 32,
 'color': 'White',
 'currency': 'GBP',
 'description': 'WELL LOOKED AFTER ,WITH FULL MAIN DEALER SERVICE HISTORY . '
                'GOOD SPEC INGLUDING SAT NAV, DAB , PARK ASSIST, REVERSING '
                "CAMERA . LINE DEPARTURE , HEATED SEATS. SUPPLIED WITH 2 KEY'S "
                'NEW MOT AND WARRANTY . PART EX AND FINANCE WELLCOME',
 'doors_number': 4,
 'emission_standard': 'Euro 6',
 'engine': '2L',
 'engine_horse_power': 288,
 'fuel_type': 'Petrol Plug-in Hybrid',
 'id': '1de69e53-e053-5042-b850-448af59d2b4c',
 'is_damaged': False,
 'is_operable': True,
 'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202603040405261',
 'listing': 'directbuy',
 'location': 'Portsmouth, Hampshire',
 'make': 'BMW',
 'model': '3 Series',
 'odometer': 100800,
 'odometer_reading_unit': 'mi',
 'origin_country_code': 'GB',
 'original_photo_count': 16,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_1de69e53-e053-5042-b850-448af59d2b4c/1de69e53-e053-5042-b850-448af59d2b4c_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 286716,
             'Height': 1152,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/32254f4d86e7433d909cf2ebfaed0e21.jpg',
             'Width': 2048}],
 'price': 14995,
 'price_includes_vat': True,
 'registration_date': '2021-03-31',
 'seats_number': 5,
 'seller_name': None,
 'title': 'BMW 3 Series 2.0 330e 12kWh M Sport Auto xDrive Euro 6 (s/s) 4dr',
 'transmission': 'automatic',
 'trim': 'M Sport',
 'year': 2021}
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603040405261%27 HTTP/1.1" 200 None
2026-03-30 12:12:04 [auto_selling_autotrader.co.uk] INFO: Saving data for 202603040405261: {'created_time': 1774872724.502181, 'last_price_update_time': 1774872724.502192}
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202603040405261') HTTP/1.1" 204 0
2026-03-30 12:12:04 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: 04d9c655-ca60-5315-9d3c-0ecf8caca08a with identifier 202603040386452
2026-03-30 12:12:04 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:04 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:04 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item 04d9c655-ca60-5315-9d3c-0ecf8caca08a
2026-03-30 12:12:04 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/758db978b19a46b79d65546735ce119d.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 121,
 'color': 'Black',
 'currency': 'GBP',
 'description': 'EXCELLENT CONDITION BOTH INSIDE AND OUT WITH A FULL SERVICE '
                'HISTORY. JUST SERVICED AT LEXUS AT 32K. GREAT SPEC INC '
                'PROTECTION PACK. SUNROOF. DRIVES WITH OUT FAULT. SUPPLIED '
                "WITH A NEW ADVISORY FREE MOT. LEXUS WARRANTY. 2KEY'S. JUST "
                'HAD 4 NEW TYRES. PART EXCHANGE WELCOME. FINANCE ARRANGED. REG '
                'DATE 29-12-22. TWO ES 300 IN STOCK.\n'
                ' 2022 Lexus ES 300h Premium Edition boasts a full dealership '
                'service history and has had only one owner from new, ensuring '
                'excellent provenance. This Euro 6 compliant model features a '
                '2.5 litre petrol hybrid engine, offering impressive fuel '
                'economy of 51 miles per gallon and low CO2 emissions. It '
                'comes with a 12 month warranty and a 12 month MOT provided on '
                'purchase, with a clear vehicle history check and the last '
                'service completed at 32,000 miles. Notable features include a '
                'Protection Pack, intelligent parking sensors all round with '
                'brake function, Bluetooth connectivity, Blind Spot Monitor '
                'with Rear Cross Traffic Alert and Auto Brake, and automatic '
                'headlights.\n'
                '\n'
                'The Lexus ES 300h Premium Edition stands out with its '
                'combination of efficiency and advanced technology. The 51 mpg '
                'fuel economy and low CO2 emissions highlight its economical '
                'and environmentally conscious performance, setting it apart '
                'from many other vehicles in its class. The inclusion of '
                'features like the Blind Spot Monitor with Rear Cross Traffic '
                'Alert and intelligent parking sensors with brake function '
                'demonstrates a commitment to driver safety and convenience, '
                'offering a more reassuring driving experience.',
 'doors_number': 4,
 'emission_standard': 'Euro 6',
 'engine': '2.5L',
 'engine_horse_power': 215,
 'fuel_type': 'Petrol Hybrid',
 'id': '04d9c655-ca60-5315-9d3c-0ecf8caca08a',
 'is_damaged': False,
 'is_operable': True,
 'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202603040386452',
 'listing': 'directbuy',
 'location': 'Portsmouth, Hampshire',
 'make': 'Lexus',
 'model': 'ES',
 'odometer': 34000,
 'odometer_reading_unit': 'mi',
 'origin_country_code': 'GB',
 'original_photo_count': 22,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_04d9c655-ca60-5315-9d3c-0ecf8caca08a/04d9c655-ca60-5315-9d3c-0ecf8caca08a_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 618592,
             'Height': 1536,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/758db978b19a46b79d65546735ce119d.jpg',
             'Width': 2048}],
 'price': 25495,
 'price_includes_vat': True,
 'registration_date': '2022-12-29',
 'seats_number': 5,
 'seller_name': None,
 'title': 'Lexus ES 2.5 300h Premium Saloon 4dr Petrol Hybrid E-CVT Euro 6 '
          '(s/s) (218 ps)',
 'transmission': 'automatic',
 'trim': 'Premium',
 'year': 2022}
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603040386452%27 HTTP/1.1" 200 None
2026-03-30 12:12:04 [auto_selling_autotrader.co.uk] INFO: Saving data for 202603040386452: {'created_time': 1774872724.554764, 'last_price_update_time': 1774872724.554774}
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202603040386452') HTTP/1.1" 204 0
2026-03-30 12:12:04 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: 284d050d-a9f7-5090-8227-72ddbb05b609 with identifier 202602190052151
2026-03-30 12:12:04 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:04 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:04 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item 284d050d-a9f7-5090-8227-72ddbb05b609
2026-03-30 12:12:04 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/4313e7b094984c218b703fb792d415a6.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 146,
 'color': 'White',
 'currency': 'GBP',
 'description': 'GREAT CONDITION FOR AGE AND MILEAGE WITH FULL MAIN DEALER '
                'SERVICE HISTORY . GOOD SPEC INCLUDING SAT NAV, PARKING '
                'SENSORS, HEATED SEATS. SUPPLIED WITH 2 KEY;S NEW MOT AND '
                'WARRANTY . This 2016 BMW X1 20i M Sport is a superb example, '
                'having been cherished by only two owners from new. It boasts '
                'a powerful 2.0 litre petrol engine paired with a smooth '
                "automatic transmission and BMW's renowned four wheel drive "
                'system, ensuring confident handling in all conditions. As a '
                'Euro 6 compliant vehicle, it meets modern emissions '
                "standards. Inside, you'll find the M Sport trim offering a "
                'sporty feel, enhanced by features like heated front seats for '
                'those colder mornings and the BMW Advanced loudspeaker system '
                'for an immersive audio experience. The Driver Comfort Package '
                'includes parking assistance with front sensors, making '
                'manoeuvring easier, while extended lighting improves '
                'visibility. Enjoy seamless connectivity with Bluetooth audio '
                'streaming for your compatible devices.\n'
                '\n'
                'The X1 20i M Sport offers impressive performance, '
                'accelerating from 0 to 62 miles per hour in just 7.4 seconds, '
                'demonstrating its dynamic capabilities. This acceleration '
                'figure highlights its advantage over many other vehicles in '
                'its class. Furthermore, the generous boot space provides '
                'ample room for luggage, shopping, or sports equipment, making '
                'it a practical choice for families and active individuals '
                'alike. The combination of its performance, practicality, and '
                'premium features makes this BMW X1 a compelling option for '
                'those seeking a versatile and engaging driving experience.',
 'doors_number': 5,
 'emission_standard': 'Euro 6',
 'engine': '2L',
 'engine_horse_power': 189,
 'fuel_type': 'Petrol',
 'id': '284d050d-a9f7-5090-8227-72ddbb05b609',
 'is_damaged': False,
 'is_operable': True,
 'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202602190052151',
 'listing': 'directbuy',
 'location': 'Portsmouth, Hampshire',
 'make': 'BMW',
 'model': 'X1',
 'odometer': 78300,
 'odometer_reading_unit': 'mi',
 'origin_country_code': 'GB',
 'original_photo_count': 24,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_284d050d-a9f7-5090-8227-72ddbb05b609/284d050d-a9f7-5090-8227-72ddbb05b609_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 416444,
             'Height': 1152,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/4313e7b094984c218b703fb792d415a6.jpg',
             'Width': 2048}],
 'price': 11995,
 'price_includes_vat': True,
 'registration_date': '2016-11-30',
 'seats_number': 5,
 'seller_name': None,
 'title': 'BMW X1 2.0 20i M Sport Auto xDrive Euro 6 (s/s) 5dr',
 'transmission': 'automatic',
 'trim': 'M Sport',
 'year': 2016}
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602190052151%27 HTTP/1.1" 200 None
2026-03-30 12:12:04 [auto_selling_autotrader.co.uk] INFO: Saving data for 202602190052151: {'created_time': 1774872724.615382, 'last_price_update_time': 1774872724.615401}
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202602190052151') HTTP/1.1" 204 0
2026-03-30 12:12:04 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/5df61fd462ba43729477492fb21a6877.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fc589ba8590>
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_c9db9e88-d61b-54b2-8dcf-3f0e6372411d/c9db9e88-d61b-54b2-8dcf-3f0e6372411d_image_0.jpg HTTP/1.1" 201 0
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F73a90cd04b1d49119443d99eb8b0fc0b.jpg') HTTP/1.1" 204 0
2026-03-30 12:12:04 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/73a90cd04b1d49119443d99eb8b0fc0b.jpg saved to cache
2026-03-30 12:12:04 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item c9db9e88-d61b-54b2-8dcf-3f0e6372411d
2026-03-30 12:12:04 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item c9db9e88-d61b-54b2-8dcf-3f0e6372411d sending to next pipeline
2026-03-30 12:12:04 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: c9db9e88-d61b-54b2-8dcf-3f0e6372411d with identifier 202602169963132
2026-03-30 12:12:04 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:04 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:04 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item c9db9e88-d61b-54b2-8dcf-3f0e6372411d
2026-03-30 12:12:04 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/73a90cd04b1d49119443d99eb8b0fc0b.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 158,
 'color': 'Black',
 'currency': 'GBP',
 'description': 'EXCELLENT CONDITION BOTH INSIDE AND OUT. GREAT SPEC CAR INC '
                'NAV. BLIND SPOT MONITER. DAB. VERY WELL LOOKED AFTER CAR WITH '
                'A FULL SERVICE HIOSTORY. DRIVES WITH OUT FAULT. SUPPLIED WITH '
                "A NEW ADVISORY FREE MOT. WARRANTY 2 KEY'S AND FULLY SERVICED. "
                'PART EXCHANGE WELCOME. FINANCE ARRANGED. \n'
                '2020 Audi Q3 TDI S line boasts an exceptionally low mileage '
                'for its age, having covered just 43,000 miles, and comes with '
                'a full service history. This Euro 6 compliant 2.0 litre '
                'diesel model offers an impressive fuel economy of 50 mpg. '
                "Inside, you'll find a flat-bottomed 3-spoke leather-trimmed "
                'multi-function sport steering wheel complete with gear-shift '
                'paddles, and supportive Alcantara Frequency-Twin leather '
                'sport seats with S embossing in black with rock grey '
                'stitching. For added convenience and safety, this Q3 includes '
                'a Bluetooth interface, Audi Side Assist for blind spot '
                'monitoring, and automatic headlights activated by a light and '
                'rain sensor.\n'
                '\n'
                'The generous boot space ensures practicality for all your '
                'needs, making this Q3 a versatile choice. The 12 months MOT '
                'provided on purchase and a 6 months warranty offer peace of '
                'mind. These features, combined with its low mileage and '
                'excellent fuel efficiency, demonstrate how this Audi Q3 '
                'stands out from other vehicles in its class, offering a '
                'compelling blend of performance, comfort, and advanced '
                'technology.',
 'doors_number': 5,
 'emission_standard': 'Euro 6',
 'engine': '2L',
 'engine_horse_power': 148,
 'fuel_type': 'Diesel',
 'id': 'c9db9e88-d61b-54b2-8dcf-3f0e6372411d',
 'is_damaged': False,
 'is_operable': True,
 'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202602169963132',
 'listing': 'directbuy',
 'location': 'Portsmouth, Hampshire',
 'make': 'Audi',
 'model': 'Q3',
 'odometer': 43000,
 'odometer_reading_unit': 'mi',
 'origin_country_code': 'GB',
 'original_photo_count': 20,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_c9db9e88-d61b-54b2-8dcf-3f0e6372411d/c9db9e88-d61b-54b2-8dcf-3f0e6372411d_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 646793,
             'Height': 1536,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/73a90cd04b1d49119443d99eb8b0fc0b.jpg',
             'Width': 2048}],
 'price': 19995,
 'price_includes_vat': True,
 'registration_date': '2020-08-14',
 'seats_number': 5,
 'seller_name': None,
 'title': 'Audi Q3 2.0 TDI 35 S line S Tronic Euro 6 (s/s) 5dr',
 'transmission': 'automatic',
 'trim': 'S line',
 'year': 2020}
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602169963132%27 HTTP/1.1" 200 None
2026-03-30 12:12:04 [auto_selling_autotrader.co.uk] INFO: Saving data for 202602169963132: {'created_time': 1774872724.748726, 'last_price_update_time': 1774872724.748738}
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202602169963132') HTTP/1.1" 204 0
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fc588303c10>
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-30 12:12:04 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_4f38df39-7ea5-512a-b214-e6b2d8f2c392/4f38df39-7ea5-512a-b214-e6b2d8f2c392_image_0.jpg HTTP/1.1" 201 0
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F5df61fd462ba43729477492fb21a6877.jpg') HTTP/1.1" 204 0
2026-03-30 12:12:04 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/5df61fd462ba43729477492fb21a6877.jpg saved to cache
2026-03-30 12:12:04 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 4f38df39-7ea5-512a-b214-e6b2d8f2c392
2026-03-30 12:12:04 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 4f38df39-7ea5-512a-b214-e6b2d8f2c392 sending to next pipeline
2026-03-30 12:12:04 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: 4f38df39-7ea5-512a-b214-e6b2d8f2c392 with identifier 202603020320093
2026-03-30 12:12:04 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:04 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:04 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item 4f38df39-7ea5-512a-b214-e6b2d8f2c392
2026-03-30 12:12:04 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/5df61fd462ba43729477492fb21a6877.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 167,
 'color': 'Grey',
 'currency': 'GBP',
 'description': 'EXCELLENT CONDITION BOTH INSIDE AND OUT WITH A FULL MERCEDES '
                'SERVICE HISTORY. JUST HAD MAJOR SERVICE. GREAT SPEC CAR. '
                'DRIVES WITHOUT FAULT. SUPPLIED WITH A NEW ADVISORY FREE MOT. '
                "WARRANTY. 2 KEY'S. PART EXCHNAGE WELCOME. FINANCE ARRANGED.\n"
                ' 2019 Mercedes-Benz A Class A35 AMG boasts a full dealership '
                'service history, with its last service completed at 74927 '
                'miles, offering you peace of mind. This Euro 6 compliant, '
                'four wheel drive model is powered by a 2.0L petrol engine and '
                'comes with a 6 month warranty and a 12 month MOT provided on '
                'purchase. The Premium Package enhances your driving '
                'experience with keyless entry, front and rear parking '
                "assistance, and heated seats. You'll also find dual Bluetooth "
                'connectivity with two USB Type C ports, a 180 degree '
                'reversing camera with parking guidelines, and LED high '
                'performance headlamps with integrated daytime running lights. '
                'Inside, enjoy the comfort of THERMOTRONIC luxury automatic '
                'climate control with two zones and a display.\n'
                '\n'
                'Experience exhilarating performance with this A35 AMG, '
                'capable of accelerating from 0 to 62mph in just 4.7 seconds, '
                'a testament to its superior engineering. This impressive '
                'acceleration, combined with its advanced features like the '
                'Premium Package and sophisticated climate control, sets it '
                'apart from other vehicles in its class. Furthermore, the '
                'generous boot space ensures practicality for all your needs, '
                'making this Mercedes-Benz A Class a well rounded and '
                'desirable choice.',
 'doors_number': 5,
 'emission_standard': 'Euro 6',
 'engine': '2L',
 'engine_horse_power': 302,
 'fuel_type': 'Petrol',
 'id': '4f38df39-7ea5-512a-b214-e6b2d8f2c392',
 'is_damaged': False,
 'is_operable': True,
 'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202603020320093',
 'listing': 'directbuy',
 'location': 'Portsmouth, Hampshire',
 'make': 'Mercedes-Benz',
 'model': 'A Class',
 'odometer': 75000,
 'odometer_reading_unit': 'mi',
 'origin_country_code': 'GB',
 'original_photo_count': 19,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_4f38df39-7ea5-512a-b214-e6b2d8f2c392/4f38df39-7ea5-512a-b214-e6b2d8f2c392_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 360646,
             'Height': 1152,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/5df61fd462ba43729477492fb21a6877.jpg',
             'Width': 2048}],
 'price': 17495,
 'price_includes_vat': True,
 'registration_date': '2019-09-26',
 'seats_number': 5,
 'seller_name': None,
 'title': 'Mercedes-Benz A Class 2.0 A35 AMG (Premium) SpdS DCT 4MATIC Euro 6 '
          '(s/s) 5dr',
 'transmission': 'automatic',
 'trim': 'AMG',
 'year': 2019}
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202603020320093%27 HTTP/1.1" 200 None
2026-03-30 12:12:04 [auto_selling_autotrader.co.uk] INFO: Saving data for 202603020320093: {'created_time': 1774872724.916812, 'last_price_update_time': 1774872724.916825}
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:04 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202603020320093') HTTP/1.1" 204 0
2026-03-30 12:12:04 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/bcae73711fd24ae9842baaf6451335d1.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-30 12:12:04 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/eb43a9b1d06a4966b129f667019b65ff.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-30 12:12:04 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://m.atcdn.co.uk/a/media/0c06b652c4484bd2933589c671a07f0e.jpg> (referer: https://www.autotrader.co.uk/)
2026-03-30 12:12:05 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-30 12:12:05 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fc589b87010>
2026-03-30 12:12:05 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-30 12:12:05 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_8d5ed4f4-e788-5732-951f-055a28676468/8d5ed4f4-e788-5732-951f-055a28676468_image_0.jpg HTTP/1.1" 201 0
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252Fbcae73711fd24ae9842baaf6451335d1.jpg') HTTP/1.1" 204 0
2026-03-30 12:12:05 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/bcae73711fd24ae9842baaf6451335d1.jpg saved to cache
2026-03-30 12:12:05 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 8d5ed4f4-e788-5732-951f-055a28676468
2026-03-30 12:12:05 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 8d5ed4f4-e788-5732-951f-055a28676468 sending to next pipeline
2026-03-30 12:12:05 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-30 12:12:05 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fc588351990>
2026-03-30 12:12:05 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-30 12:12:05 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_46c48132-41d5-51d7-8155-26e97a8e0b8a/46c48132-41d5-51d7-8155-26e97a8e0b8a_image_0.jpg HTTP/1.1" 201 0
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252Feb43a9b1d06a4966b129f667019b65ff.jpg') HTTP/1.1" 204 0
2026-03-30 12:12:05 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/eb43a9b1d06a4966b129f667019b65ff.jpg saved to cache
2026-03-30 12:12:05 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item 46c48132-41d5-51d7-8155-26e97a8e0b8a
2026-03-30 12:12:05 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item 46c48132-41d5-51d7-8155-26e97a8e0b8a sending to next pipeline
2026-03-30 12:12:05 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: 8d5ed4f4-e788-5732-951f-055a28676468 with identifier 202602250206482
2026-03-30 12:12:05 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:05 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:05 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item 8d5ed4f4-e788-5732-951f-055a28676468
2026-03-30 12:12:05 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/bcae73711fd24ae9842baaf6451335d1.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 182,
 'color': 'Silver',
 'currency': 'GBP',
 'description': 'EXCELLENT CONDITION BOTH INSIDE AND OUT WITH A FULL LANDROVER '
                'MAIN DEALER SERVICE HISTORY. LAST 14/05/2025. OVER 10K OF '
                "OPTION'S INC HEAD UP DISPLAY. 360 CAMERA. SOFT CLOSE "
                "DOOR'S.OPENING PAN ROOF. 22'' ALLOY'S. DRIVES WITH OUT FAULT. "
                'SUPPLIED WITH A NEW ADISORY FREE MOT. WARRANTY AND FULLY '
                "SERVICED. 2 KEY'S. PART EXCHANGE WELCOME. FINANCE ARRANGED. I "
                'WOULD SAY METALLIC GREY NOT SILVER.\n'
                ' 2018 Land Rover Range Rover TD V6 Vogue boasts a full '
                'dealership service history and a low mileage of just 66,000 '
                'miles for its age, ensuring excellent provenance. This '
                'luxurious Vogue trim features a powerful 3.0L Diesel engine, '
                'a Four Wheel Drive drivetrain, and meets Euro 6 emissions '
                'standards. Notable features include a Head Up Display for '
                'convenient information access, a 360 Degree Surround Camera '
                'System for effortless parking assistance, and a premium '
                'Meridian Surround Sound System for an immersive audio '
                'experience. The striking Black Exterior Pack and Contrast '
                'Painted Roof in Narvik Black add to its sophisticated '
                'appearance.\n'
                '\n'
                'Experience impressive performance with an acceleration of '
                '0-60mph in just 7.5 seconds, a testament to its capable 3.0L '
                'Diesel engine. This Range Rover offers a substantial towing '
                'weight of 3500 kg, making it ideal for various needs, and '
                'benefits from a generous boot space for all your luggage. A 6 '
                'months warranty and 12 months MOT provided on purchase offer '
                'added peace of mind. These highlights demonstrate how this '
                'particular Range Rover stands out from others in its class.',
 'doors_number': 5,
 'emission_standard': 'Euro 6',
 'engine': '3L',
 'engine_horse_power': 254,
 'fuel_type': 'Diesel',
 'id': '8d5ed4f4-e788-5732-951f-055a28676468',
 'is_damaged': False,
 'is_operable': True,
 'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202602250206482',
 'listing': 'directbuy',
 'location': 'Portsmouth, Hampshire',
 'make': 'Land Rover',
 'model': 'Range Rover',
 'odometer': 66000,
 'odometer_reading_unit': 'mi',
 'origin_country_code': 'GB',
 'original_photo_count': 26,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_8d5ed4f4-e788-5732-951f-055a28676468/8d5ed4f4-e788-5732-951f-055a28676468_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 534224,
             'Height': 1536,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/bcae73711fd24ae9842baaf6451335d1.jpg',
             'Width': 2048}],
 'price': 28495,
 'price_includes_vat': True,
 'registration_date': '2018-03-08',
 'seats_number': 5,
 'seller_name': None,
 'title': 'Land Rover Range Rover 3.0 TD V6 Vogue Auto 4WD Euro 6 (s/s) 5dr',
 'transmission': 'automatic',
 'trim': 'Vogue',
 'year': 2018}
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602250206482%27 HTTP/1.1" 200 None
2026-03-30 12:12:05 [auto_selling_autotrader.co.uk] INFO: Saving data for 202602250206482: {'created_time': 1774872725.300618, 'last_price_update_time': 1774872725.300627}
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202602250206482') HTTP/1.1" 204 0
2026-03-30 12:12:05 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: 46c48132-41d5-51d7-8155-26e97a8e0b8a with identifier 202602200085700
2026-03-30 12:12:05 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:05 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:05 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item 46c48132-41d5-51d7-8155-26e97a8e0b8a
2026-03-30 12:12:05 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/eb43a9b1d06a4966b129f667019b65ff.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 124,
 'color': 'Grey',
 'currency': 'GBP',
 'description': 'WELL LOOKED AFTER WITH FULL MAIN DEALER SERVICE HISTORY . '
                'JUST HAD A NEW CAM BELT AND WATER PUMP . HIGH SPEC INCLUDING '
                'SAT NAV, DAB , REVESING CAMERA , LINE DEPARTURE , BLIND SPOT '
                'MONITOR , DISTANCE CRUISE CONTROL ,PANAROMIC SUNROOF '
                ".SUPPLIED WITH 2 KEY'S NEW MOT AND WARRANTY , PART EX AND "
                'FINANCE WELLCOME .',
 'doors_number': 5,
 'emission_standard': 'Euro 6',
 'engine': '2L',
 'engine_horse_power': 175,
 'fuel_type': 'Diesel',
 'id': '46c48132-41d5-51d7-8155-26e97a8e0b8a',
 'is_damaged': False,
 'is_operable': True,
 'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202602200085700',
 'listing': 'directbuy',
 'location': 'Portsmouth, Hampshire',
 'make': 'Citroen',
 'model': 'C5 Aircross',
 'odometer': 95000,
 'odometer_reading_unit': 'mi',
 'origin_country_code': 'GB',
 'original_photo_count': 19,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_46c48132-41d5-51d7-8155-26e97a8e0b8a/46c48132-41d5-51d7-8155-26e97a8e0b8a_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 453520,
             'Height': 1152,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/eb43a9b1d06a4966b129f667019b65ff.jpg',
             'Width': 2048}],
 'price': 9995,
 'price_includes_vat': True,
 'registration_date': '2019-09-20',
 'seats_number': 5,
 'seller_name': None,
 'title': 'Citroen C5 Aircross 2.0 BlueHDi Flair Plus EAT8 Euro 6 (s/s) 5dr',
 'transmission': 'automatic',
 'trim': 'Flair Plus',
 'year': 2019}
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602200085700%27 HTTP/1.1" 200 None
2026-03-30 12:12:05 [auto_selling_autotrader.co.uk] INFO: Saving data for 202602200085700: {'created_time': 1774872725.348519, 'last_price_update_time': 1774872725.348529}
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202602200085700') HTTP/1.1" 204 0
2026-03-30 12:12:05 [common.azure_blobstore] INFO: Creating container client for top-deal-listing
2026-03-30 12:12:05 [common.azure_blobstore] INFO: Container_client: <azure.storage.blob._container_client.ContainerClient object at 0x7fc58833c2d0>
2026-03-30 12:12:05 [common.azure_blobstore] INFO: Creating container if not exists
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing?restype=container HTTP/1.1" 409 230
2026-03-30 12:12:05 [common.azure_blobstore] INFO: Container already exists, skipping creation
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxproduction.blob.core.windows.net:443 "PUT /top-deal-listing/listing_dc2156ba-529d-59be-b5c5-fb93cbff1885/dc2156ba-529d-59be-b5c5-fb93cbff1885_image_0.jpg HTTP/1.1" 201 0
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /PhotoDownloadCache(PartitionKey='auto_selling_autotrader.co.uk',RowKey='https%3A%252F%252Fm.atcdn.co.uk%252Fa%252Fmedia%252F0c06b652c4484bd2933589c671a07f0e.jpg') HTTP/1.1" 204 0
2026-03-30 12:12:05 [crawlers.middlewares.photo_download_middleware] INFO: Image https://m.atcdn.co.uk/a/media/0c06b652c4484bd2933589c671a07f0e.jpg saved to cache
2026-03-30 12:12:05 [crawlers.middlewares.photo_download_middleware] INFO: Downloaded image 0 for item dc2156ba-529d-59be-b5c5-fb93cbff1885
2026-03-30 12:12:05 [crawlers.middlewares.photo_download_middleware] INFO: All photos downloaded for item dc2156ba-529d-59be-b5c5-fb93cbff1885 sending to next pipeline
2026-03-30 12:12:05 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: dc2156ba-529d-59be-b5c5-fb93cbff1885 with identifier 202602200074354
2026-03-30 12:12:05 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:05 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:05 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item dc2156ba-529d-59be-b5c5-fb93cbff1885
2026-03-30 12:12:05 [scrapy.core.scraper] DEBUG: Scraped from <200 https://m.atcdn.co.uk/a/media/0c06b652c4484bd2933589c671a07f0e.jpg>
{'auction_title': '',
 'c_o_2_emission_value': 142,
 'color': 'Grey',
 'currency': 'GBP',
 'description': 'TOTALLY UNMARKED BOTH INSIDE AND OUT WITH A FULL SERVICE '
                'HISTORY. GREAT SPEC INC SATNAV. LEATHER. PAN ROOF. DRIVES '
                'WITH OUT FAULT. SUPPLIED WITH A NEW ADVISORY FREE MOT. MAIN '
                'DEALER WARRANTY UNTIL. 30/06/2026. GREAT FINANCE OPTIONS '
                'AVAILABLE. PART EXCHANGE WELCOME.  2023 Citroen C5 Aircross C '
                'Series Edition boasts a full service history, with its last '
                'service completed at 20,800 miles, offering you peace of '
                'mind. This stylish SUV is powered by an efficient 1.2 litre '
                'petrol engine, meeting Euro 6 emissions standards. It comes '
                'with a 12 month warranty and a 12 month MOT provided on '
                'purchase, ensuring your investment is protected. Notable '
                'features include the Drive Assist Pack, which incorporates '
                'Blind Spot Assistance and Adaptive Cruise Control, enhancing '
                "your driving experience. You'll also benefit from an acoustic "
                'laminated windscreen with rain and brightness sensors, '
                'automatic headlights with welcome and goodbye lighting, and '
                'convenient wireless smartphone charging. The electric, '
                'heated, and folding door mirrors are equipped with LED '
                'approach lighting and blind spot monitoring indicators.\n'
                '\n'
                'Experience impressive fuel economy with this C5 Aircross, '
                'achieving a highlight of 49 miles per gallon, which is better '
                'than many similar vehicles. Its low CO2 emissions also '
                'contribute to its efficiency. Furthermore, this model offers '
                'generous boot space, making it a practical choice for '
                'families and those who need ample room for luggage or '
                'shopping. The combination of its advanced features, '
                'economical performance, and spacious interior makes this '
                'Citroen C5 Aircross C Series Edition a compelling option.',
 'doors_number': 5,
 'emission_standard': 'Euro 6',
 'engine': '1.2L',
 'engine_horse_power': 128,
 'fuel_type': 'Petrol',
 'id': 'dc2156ba-529d-59be-b5c5-fb93cbff1885',
 'is_damaged': False,
 'is_operable': True,
 'link_to_web_offer': 'https://www.autotrader.co.uk/car-details/202602200074354',
 'listing': 'directbuy',
 'location': 'Portsmouth, Hampshire',
 'make': 'Citroen',
 'model': 'C5 Aircross',
 'odometer': 23327,
 'odometer_reading_unit': 'mi',
 'origin_country_code': 'GB',
 'original_photo_count': 24,
 'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_dc2156ba-529d-59be-b5c5-fb93cbff1885/dc2156ba-529d-59be-b5c5-fb93cbff1885_image_0.jpg',
             'Extension': 'jpg',
             'FileSizeInBytes': 558896,
             'Height': 1536,
             'OriginUrl': 'https://m.atcdn.co.uk/a/media/0c06b652c4484bd2933589c671a07f0e.jpg',
             'Width': 2048}],
 'price': 14995,
 'price_includes_vat': True,
 'registration_date': '2023-06-30',
 'seats_number': 5,
 'seller_name': None,
 'title': 'Citroen C5 Aircross 1.2 PureTech C-Series Edition Euro 6 (s/s) 5dr',
 'transmission': 'manual',
 'trim': 'C-Series Edition',
 'year': 2023}
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20RowKey%20eq%20%27202602200074354%27 HTTP/1.1" 200 None
2026-03-30 12:12:05 [auto_selling_autotrader.co.uk] INFO: Saving data for 202602200074354: {'created_time': 1774872725.468534, 'last_price_update_time': 1774872725.468543}
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_autotrader.co.uk',RowKey='202602200074354') HTTP/1.1" 204 0
2026-03-30 12:12:05 [scrapy.core.engine] INFO: Closing spider (finished)
2026-03-30 12:12:05 [crawlers.pipelines.save_to_cdlq_db_auto_selling] DEBUG: [SaveToCdlqDbAutoSelling] failed to close cursor: 'SaveToCdlqDbAutoSelling' object has no attribute 'cursor'
2026-03-30 12:12:05 [crawlers.pipelines.save_to_cdlq_db_auto_selling] DEBUG: [SaveToCdlqDbAutoSelling] failed to close connection: 'SaveToCdlqDbAutoSelling' object has no attribute 'conn'
2026-03-30 12:12:05 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:05 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-30 12:12:05 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process scrape complete CloudEvent for item 93ca90c02c3111f1b320c264925fbcfa
2026-03-30 12:12:05 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2026-03-30 12:12:05 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2026-03-30 12:12:05 [azure.servicebus._pyamqp.management_link] INFO: Management link receiver state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2026-03-30 12:12:05 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2026-03-30 12:12:05 [azure.servicebus._pyamqp.management_link] INFO: Management link sender state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2026-03-30 12:12:05 [azure.servicebus._pyamqp.session] INFO: Session state changed: <SessionState.MAPPED: 3> -> <SessionState.END_SENT: 4>
2026-03-30 12:12:05 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.OPENED: 9> -> <ConnectionState.CLOSE_SENT: 11>
2026-03-30 12:12:05 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.CLOSE_SENT: 11> -> <ConnectionState.END: 13>
2026-03-30 12:12:05 [azure.servicebus._pyamqp.session] INFO: Session state changed: <SessionState.END_SENT: 4> -> <SessionState.DISCARDING: 6>
2026-03-30 12:12:05 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2026-03-30 12:12:05 [azure.servicebus._pyamqp.management_link] INFO: Management link sender state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2026-03-30 12:12:05 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2026-03-30 12:12:05 [azure.servicebus._pyamqp.management_link] INFO: Management link receiver state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2026-03-30 12:12:05 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2026-03-30 12:12:05 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: [AzureBusPipeline] connection closed
2026-03-30 12:12:05 [auto_selling_autotrader.co.uk] INFO: auto_selling_autotrader.co.uk Crawl ended with reason finished, scrape types: {<ScrapeType.NEW: 1>: 0, <ScrapeType.NEW_DUPLICATE_ID: 4>: 0, <ScrapeType.PRICE_UPDATE: 2>: 0, <ScrapeType.AUCTION_UPDATE: 3>: 0, <ScrapeType.SKIPPED: 0>: 0}
2026-03-30 12:12:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-30 12:12:06 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_autotrader.co.uk%27%20and%20last_price_update_time%20lt%201773576726 HTTP/1.1" 200 None
2026-03-30 12:12:06 [scrapy.extensions.feedexport] INFO: Stored jsonlines feed (15 items) in: file:///var/lib/scrapyd/items/sourcing_v2/auto_selling_autotrader.co.uk/93ca90c02c3111f1b320c264925fbcfa.jl
2026-03-30 12:12:06 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 122320,
 'downloader/request_count': 18,
 'downloader/request_method_count/GET': 15,
 'downloader/request_method_count/POST': 3,
 'downloader/response_bytes': 7474975,
 'downloader/response_count': 18,
 'downloader/response_status_count/200': 18,
 'elapsed_time_seconds': 33.271733,
 'feedexport/success_count/FileFeedStorage': 1,
 'finish_reason': 'finished',
 'finish_time': datetime.datetime(2026, 3, 30, 12, 12, 6, 481381, tzinfo=datetime.timezone.utc),
 'item_scraped_count': 15,
 'log_count/DEBUG': 287,
 'log_count/INFO': 274,
 'memusage/max': 147009536,
 'memusage/startup': 147009536,
 'photo_download_count': 15,
 'request_depth_max': 2,
 'response_received_count': 3,
 'scheduler/dequeued': 18,
 'scheduler/dequeued/memory': 18,
 'scheduler/enqueued': 18,
 'scheduler/enqueued/memory': 18,
 'scrape_type/new': 15,
 'scrapy-zyte-api/429': 0,
 'scrapy-zyte-api/attempts': 3,
 'scrapy-zyte-api/error_ratio': 0.0,
 'scrapy-zyte-api/errors': 0,
 'scrapy-zyte-api/fatal_errors': 0,
 'scrapy-zyte-api/mean_connection_seconds': 7.746845664301267,
 'scrapy-zyte-api/mean_response_seconds': 8.345861734667173,
 'scrapy-zyte-api/processed': 3,
 'scrapy-zyte-api/request_args/customHttpRequestHeaders': 3,
 'scrapy-zyte-api/request_args/experimental.requestCookies': 2,
 'scrapy-zyte-api/request_args/experimental.responseCookies': 3,
 'scrapy-zyte-api/request_args/httpRequestBody': 3,
 'scrapy-zyte-api/request_args/httpRequestMethod': 3,
 'scrapy-zyte-api/request_args/httpResponseBody': 3,
 'scrapy-zyte-api/request_args/httpResponseHeaders': 3,
 'scrapy-zyte-api/request_args/url': 3,
 'scrapy-zyte-api/status_codes/200': 3,
 'scrapy-zyte-api/success': 3,
 'scrapy-zyte-api/success_ratio': 1.0,
 'scrapy-zyte-api/throttle_ratio': 0.0,
 'start_time': datetime.datetime(2026, 3, 30, 12, 11, 33, 209648, tzinfo=datetime.timezone.utc)}
2026-03-30 12:12:06 [scrapy.core.engine] INFO: Spider closed (finished)
2026-03-30 12:12:06 [asyncio] ERROR: Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7fc58b2e1f50>
2026-03-30 12:12:06 [asyncio] ERROR: Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7fc58b2e3d90>