2026-04-01 07:47:06 [scrapy.utils.log] INFO: Scrapy 2.11.2 started (bot: SourcingV2)
2026-04-01 07:47:06 [scrapy.utils.log] INFO: Versions: lxml 5.2.2.0, libxml2 2.12.6, cssselect 1.2.0, parsel 1.9.1, w3lib 2.1.2, Twisted 24.3.0, Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0], pyOpenSSL 24.1.0 (OpenSSL 3.2.1 30 Jan 2024), cryptography 42.0.7, Platform Linux-5.15.0-1098-azure-x86_64-with-glibc2.36
2026-04-01 07:47:06 [autobid.de] INFO: Force full scrape enabled via spider argument
2026-04-01 07:47:06 [scrapy.addons] INFO: Enabled addons:
[]
2026-04-01 07:47:06 [asyncio] DEBUG: Using selector: EpollSelector
2026-04-01 07:47:06 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.asyncioreactor.AsyncioSelectorReactor
2026-04-01 07:47:06 [scrapy.utils.log] DEBUG: Using asyncio event loop: asyncio.unix_events._UnixSelectorEventLoop
2026-04-01 07:47:06 [scrapy.extensions.telnet] INFO: Telnet Password: c21e1d4473800b2f
2026-04-01 07:47:07 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.memusage.MemoryUsage',
 'scrapy.extensions.feedexport.FeedExporter',
 'scrapy.extensions.logstats.LogStats',
 'scrapy.extensions.closespider.CloseSpider']
2026-04-01 07:47:07 [scrapy.crawler] INFO: Overridden settings:
{'BOT_NAME': 'SourcingV2',
 'CLOSESPIDER_TIMEOUT': 7200,
 'DOWNLOAD_DELAY': 1,
 'DOWNLOAD_MAXSIZE': 52428800,
 'DOWNLOAD_WARNSIZE': 10485760,
 'FEED_EXPORT_ENCODING': 'utf-8',
 'LOG_FILE': '/var/log/scrapyd/logs/sourcing_v2/autobid.de/2026-04-01T09_46_38.log',
 'LOG_FORMATTER': 'crawlers.log_formatter.SourcingLogFormatter',
 'MEMUSAGE_LIMIT_MB': 2048,
 'MEMUSAGE_WARNING_MB': 1536,
 'NEWSPIDER_MODULE': 'spiders',
 'REQUEST_FINGERPRINTER_CLASS': 'scrapy_zyte_api.ScrapyZyteAPIRequestFingerprinter',
 'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7',
 'SPIDER_MODULES': ['spiders', 'auth_check'],
 'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor',
 'USER_AGENT': ''}
2026-04-01 07:47:07 [scrapy_zyte_api.handler] INFO: Using a Zyte API key starting with '5857011'
2026-04-01 07:47:07 [scrapy_zyte_api.handler] INFO: Using a Zyte API key starting with '5857011'
2026-04-01 07:47:07 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
 'scrapy_zyte_api.ScrapyZyteAPIDownloaderMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
 'scrapy.downloadermiddlewares.stats.DownloaderStats']
2026-04-01 07:47:07 [crawlers.middlewares.id_gen_middleware] INFO: Setting up IdGenerationMiddleware
2026-04-01 07:47:07 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
 'scrapy_zyte_api.ScrapyZyteAPISpiderMiddleware',
 'crawlers.middlewares.monitoring_spider_middleware.MonitoringSpiderMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware',
 'crawlers.middlewares.photo_download_middleware.PhotoDownloadMiddleware',
 'crawlers.middlewares.report_download_middleware.ReportDownloadMiddleware',
 'crawlers.middlewares.id_gen_middleware.IdGenMiddleware']
2026-04-01 07:47:07 [azure.identity._credentials.environment] INFO: Incomplete environment configuration for EnvironmentCredential. These variables are set: AZURE_TENANT_ID, AZURE_CLIENT_ID
2026-04-01 07:47:07 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2026-04-01 07:47:07 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 2114
2026-04-01 07:47:07 [azure.identity._credentials.chained] INFO: DefaultAzureCredential acquired a token from WorkloadIdentityCredential
2026-04-01 07:47:07 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxsourcingstorageprod.table.core.windows.net:443
2026-04-01 07:47:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:07 [crawlers.pipelines.translation_pipeline] INFO: Loading translations for language: auto
2026-04-01 07:47:07 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /Translations()?$filter=PartitionKey%20eq%20%27auto%27%20and%20RowKey%20eq%20%27auto%27 HTTP/1.1" 200 None
2026-04-01 07:47:07 [crawlers.pipelines.item_rules_pipeline] INFO: Setting up ItemRules Pipeline
2026-04-01 07:47:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: invalid_location_for_country.json
2026-04-01 07:47:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: damaged_cars_from_auction_title.json
2026-04-01 07:47:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: puretech_for_country.json
2026-04-01 07:47:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: puretech_for_fr.json
2026-04-01 07:47:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: missing_photos.json
2026-04-01 07:47:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: damaged_from_info.json
2026-04-01 07:47:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: makes_not_allowed.json
2026-04-01 07:47:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: not_operable_from_info.json
2026-04-01 07:47:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: makes_models_not_allowed.json
2026-04-01 07:47:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: keywords_from_title.json
2026-04-01 07:47:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: imported_cars.json
2026-04-01 07:47:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: invalid_currency.json
2026-04-01 07:47:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: missing_mileage.json
2026-04-01 07:47:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: keywords_from_auction_title.json
2026-04-01 07:47:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: invalid_country_of_origin.json
2026-04-01 07:47:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: puretech_for_pt.json
2026-04-01 07:47:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: electric_cars.json
2026-04-01 07:47:07 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: missing_color.json
2026-04-01 07:47:07 [crawlers.pipelines.post_to_api] INFO: Setting up PostToApi Pipeline pointing to https://api.app.infinit.cc/api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing
2026-04-01 07:47:07 [scrapy.middleware] INFO: Enabled item pipelines:
['crawlers.pipelines.translation_pipeline.TranslationPipeline',
 'crawlers.pipelines.item_rules_pipeline.ItemRulesPipeline',
 'crawlers.pipelines.post_to_api.PostToApiPipeline']
2026-04-01 07:47:07 [scrapy.core.engine] INFO: Spider opened
2026-04-01 07:47:07 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 07:47:07 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2026-04-01 07:47:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 140MiB
2026-04-01 07:47:07 [scrapy-playwright] INFO: Starting download handler
2026-04-01 07:47:07 [scrapy-playwright] INFO: Starting download handler
2026-04-01 07:47:15 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=1> (referer: None)
2026-04-01 07:47:15 [autobid.de] INFO: Page 1/143: found 20 items
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360080%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [autobid.de] INFO: Scrape type for 3360080: 1
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360079%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [autobid.de] INFO: Scrape type for 3360079: 1
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359679%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [autobid.de] INFO: Scrape type for 3359679: 1
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360084%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [autobid.de] INFO: Scrape type for 3360084: 1
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355036%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [autobid.de] INFO: Scrape type for 3355036: 1
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360083%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [autobid.de] INFO: Scrape type for 3360083: 1
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360077%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [autobid.de] INFO: Scrape type for 3360077: 1
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362669%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [autobid.de] INFO: Scrape type for 3362669: 1
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362691%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [autobid.de] INFO: Scrape type for 3362691: 1
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362676%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [autobid.de] INFO: Scrape type for 3362676: 1
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357581%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [autobid.de] INFO: Scrape type for 3357581: 1
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352621%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [autobid.de] INFO: Scrape type for 3352621: 1
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362683%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [autobid.de] INFO: Scrape type for 3362683: 1
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362695%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [autobid.de] INFO: Scrape type for 3362695: 1
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362682%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [autobid.de] INFO: Scrape type for 3362682: 1
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362699%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [autobid.de] INFO: Scrape type for 3362699: 1
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362663%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [autobid.de] INFO: Scrape type for 3362663: 1
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362680%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [autobid.de] INFO: Scrape type for 3362680: 1
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362684%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [autobid.de] INFO: Scrape type for 3362684: 1
2026-04-01 07:47:15 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-a4-avant-35-tdi-s-tronic-3360080/details> (referer: None)
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362931%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [autobid.de] INFO: Scrape type for 3362931: 1
2026-04-01 07:47:15 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 3566bbdb-3e02-578f-a4fb-09d37897e274 with identifier: 3360080
2026-04-01 07:47:15 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 3566bbdb-3e02-578f-a4fb-09d37897e274 with scrape type 1
2026-04-01 07:47:15 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 3566bbdb-3e02-578f-a4fb-09d37897e274
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360080%252F93590984_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360080/93590984_hd.jpg not downloaded yet
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360080%252F93590985_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360080/93590985_hd.jpg not downloaded yet
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360080%252F93590986_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360080/93590986_hd.jpg not downloaded yet
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360080%252F93590987_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360080/93590987_hd.jpg not downloaded yet
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:15 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360080%252F93590988_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:15 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360080/93590988_hd.jpg not downloaded yet
2026-04-01 07:47:16 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/skoda-octavia-combi-1-5-tsi-ambition-3360079/details> (referer: None)
2026-04-01 07:47:16 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 4f0de064-e711-5575-9166-3d18fba5bf5d with identifier: 3360079
2026-04-01 07:47:16 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 4f0de064-e711-5575-9166-3d18fba5bf5d with scrape type 1
2026-04-01 07:47:16 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 4f0de064-e711-5575-9166-3d18fba5bf5d
2026-04-01 07:47:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360079%252F93588685_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:16 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360079/93588685_hd.jpg not downloaded yet
2026-04-01 07:47:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360079%252F93588686_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:16 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360079/93588686_hd.jpg not downloaded yet
2026-04-01 07:47:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360079%252F93588687_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:16 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360079/93588687_hd.jpg not downloaded yet
2026-04-01 07:47:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360079%252F93588688_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:16 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360079/93588688_hd.jpg not downloaded yet
2026-04-01 07:47:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360079%252F93588689_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:16 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360079/93588689_hd.jpg not downloaded yet
2026-04-01 07:47:17 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-passat-variant-2-0-tdi-bmt-dsg-comfortline-3359679/details> (referer: None)
2026-04-01 07:47:18 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 5c5b7dc0-0d5d-5a15-9dda-d2e5b8cc16ce with identifier: 3359679
2026-04-01 07:47:18 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 5c5b7dc0-0d5d-5a15-9dda-d2e5b8cc16ce with scrape type 1
2026-04-01 07:47:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 5c5b7dc0-0d5d-5a15-9dda-d2e5b8cc16ce
2026-04-01 07:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33597%252F3359679%252F93534561_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33597/3359679/93534561_hd.jpg not downloaded yet
2026-04-01 07:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33597%252F3359679%252F93534562_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33597/3359679/93534562_hd.jpg not downloaded yet
2026-04-01 07:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33597%252F3359679%252F93534576_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33597/3359679/93534576_hd.jpg not downloaded yet
2026-04-01 07:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33597%252F3359679%252F93534569_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33597/3359679/93534569_hd.jpg not downloaded yet
2026-04-01 07:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33597%252F3359679%252F93534563_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33597/3359679/93534563_hd.jpg not downloaded yet
2026-04-01 07:47:18 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ford-mondeo-2-0-tdci-powershift-titanium-3360084/details> (referer: None)
2026-04-01 07:47:19 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: e2cc5ff5-2036-50e5-a4de-52490ea215b5 with identifier: 3360084
2026-04-01 07:47:19 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e2cc5ff5-2036-50e5-a4de-52490ea215b5 with scrape type 1
2026-04-01 07:47:19 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item e2cc5ff5-2036-50e5-a4de-52490ea215b5
2026-04-01 07:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360084%252F93591132_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360084/93591132_hd.jpg not downloaded yet
2026-04-01 07:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360084%252F93591133_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360084/93591133_hd.jpg not downloaded yet
2026-04-01 07:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360084%252F93591134_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360084/93591134_hd.jpg not downloaded yet
2026-04-01 07:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360084%252F93591136_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360084/93591136_hd.jpg not downloaded yet
2026-04-01 07:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360084%252F93591137_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360084/93591137_hd.jpg not downloaded yet
2026-04-01 07:47:20 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-tiguan-2-0-tdi-bmt-team-3355036/details> (referer: None)
2026-04-01 07:47:20 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: e67f63f0-717d-5212-8025-24cb206fd0ea with identifier: 3355036
2026-04-01 07:47:20 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e67f63f0-717d-5212-8025-24cb206fd0ea with scrape type 1
2026-04-01 07:47:20 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item e67f63f0-717d-5212-8025-24cb206fd0ea
2026-04-01 07:47:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33551%252F3355036%252F93607629_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33551/3355036/93607629_hd.jpg not downloaded yet
2026-04-01 07:47:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33551%252F3355036%252F93607639_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33551/3355036/93607639_hd.jpg not downloaded yet
2026-04-01 07:47:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33551%252F3355036%252F93607647_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33551/3355036/93607647_hd.jpg not downloaded yet
2026-04-01 07:47:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33551%252F3355036%252F93607643_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33551/3355036/93607643_hd.jpg not downloaded yet
2026-04-01 07:47:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33551%252F3355036%252F93607640_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33551/3355036/93607640_hd.jpg not downloaded yet
2026-04-01 07:47:21 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-a5-cabriolet-1-8-tfsi-multitronic-s-line-3360083/details> (referer: None)
2026-04-01 07:47:21 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 80ec52c9-52e5-539b-9bc4-562765b36062 with identifier: 3360083
2026-04-01 07:47:21 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 80ec52c9-52e5-539b-9bc4-562765b36062 with scrape type 1
2026-04-01 07:47:21 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 80ec52c9-52e5-539b-9bc4-562765b36062
2026-04-01 07:47:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360083%252F93588741_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360083/93588741_hd.jpg not downloaded yet
2026-04-01 07:47:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360083%252F93588742_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360083/93588742_hd.jpg not downloaded yet
2026-04-01 07:47:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360083%252F93588743_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360083/93588743_hd.jpg not downloaded yet
2026-04-01 07:47:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360083%252F93588744_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360083/93588744_hd.jpg not downloaded yet
2026-04-01 07:47:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360083%252F93588745_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360083/93588745_hd.jpg not downloaded yet
2026-04-01 07:47:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/skoda-fabia-1-2-tsi-dsg-family-3360077/details> (referer: None)
2026-04-01 07:47:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 1cb44775-5daf-5d7a-9fbe-a9ac60c621d2 with identifier: 3360077
2026-04-01 07:47:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 1cb44775-5daf-5d7a-9fbe-a9ac60c621d2 with scrape type 1
2026-04-01 07:47:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 1cb44775-5daf-5d7a-9fbe-a9ac60c621d2
2026-04-01 07:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360077%252F93591086_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360077/93591086_hd.jpg not downloaded yet
2026-04-01 07:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360077%252F93591087_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360077/93591087_hd.jpg not downloaded yet
2026-04-01 07:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360077%252F93591088_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360077/93591088_hd.jpg not downloaded yet
2026-04-01 07:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360077%252F93591089_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360077/93591089_hd.jpg not downloaded yet
2026-04-01 07:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360077%252F93591090_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360077/93591090_hd.jpg not downloaded yet
2026-04-01 07:47:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-a6-avant-2-0-tfsi-quattro-s-tronic-3362669/details> (referer: None)
2026-04-01 07:47:24 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: f0da4c24-4ce3-50bc-ae1e-2f038adb2ba6 with identifier: 3362669
2026-04-01 07:47:24 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f0da4c24-4ce3-50bc-ae1e-2f038adb2ba6 with scrape type 1
2026-04-01 07:47:24 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item f0da4c24-4ce3-50bc-ae1e-2f038adb2ba6
2026-04-01 07:47:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362669%252F93482459_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362669/93482459_hd.jpg not downloaded yet
2026-04-01 07:47:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362669%252F93482457_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362669/93482457_hd.jpg not downloaded yet
2026-04-01 07:47:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362669%252F93482467_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362669/93482467_hd.jpg not downloaded yet
2026-04-01 07:47:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362669%252F93482475_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362669/93482475_hd.jpg not downloaded yet
2026-04-01 07:47:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362669%252F93482479_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362669/93482479_hd.jpg not downloaded yet
2026-04-01 07:47:25 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/skoda-karoq-2-0-tdi-4x4-dsg-style-3362691/details> (referer: None)
2026-04-01 07:47:25 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 2595041c-3131-5750-8b11-40d3e633f77f with identifier: 3362691
2026-04-01 07:47:25 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 2595041c-3131-5750-8b11-40d3e633f77f with scrape type 1
2026-04-01 07:47:25 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 2595041c-3131-5750-8b11-40d3e633f77f
2026-04-01 07:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362691%252F93483354_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362691/93483354_hd.jpg not downloaded yet
2026-04-01 07:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362691%252F93483355_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362691/93483355_hd.jpg not downloaded yet
2026-04-01 07:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362691%252F93483360_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362691/93483360_hd.jpg not downloaded yet
2026-04-01 07:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362691%252F93483367_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362691/93483367_hd.jpg not downloaded yet
2026-04-01 07:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362691%252F93483366_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362691/93483366_hd.jpg not downloaded yet
2026-04-01 07:47:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/skoda-kodiaq-2-0-tdi-4x4-style-3362676/details> (referer: None)
2026-04-01 07:47:26 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 9596939d-952c-5ba8-87f8-831d38662e87 with identifier: 3362676
2026-04-01 07:47:26 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9596939d-952c-5ba8-87f8-831d38662e87 with scrape type 1
2026-04-01 07:47:26 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 9596939d-952c-5ba8-87f8-831d38662e87
2026-04-01 07:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362676%252F93482769_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362676/93482769_hd.jpg not downloaded yet
2026-04-01 07:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362676%252F93482756_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362676/93482756_hd.jpg not downloaded yet
2026-04-01 07:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362676%252F93482768_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362676/93482768_hd.jpg not downloaded yet
2026-04-01 07:47:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362676%252F93482755_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362676/93482755_hd.jpg not downloaded yet
2026-04-01 07:47:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362676%252F93482767_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362676/93482767_hd.jpg not downloaded yet
2026-04-01 07:47:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-golf-sportsvan-1-6-tdi-bmt-dsg-comfortline-3357581/details> (referer: None)
2026-04-01 07:47:28 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 066652a8-7812-57ae-ab95-81add126dea4 with identifier: 3357581
2026-04-01 07:47:28 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 066652a8-7812-57ae-ab95-81add126dea4 with scrape type 1
2026-04-01 07:47:28 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 066652a8-7812-57ae-ab95-81add126dea4
2026-04-01 07:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357581%252F93454476_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357581/93454476_hd.jpg not downloaded yet
2026-04-01 07:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357581%252F93454477_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357581/93454477_hd.jpg not downloaded yet
2026-04-01 07:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357581%252F93454478_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357581/93454478_hd.jpg not downloaded yet
2026-04-01 07:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357581%252F93454479_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357581/93454479_hd.jpg not downloaded yet
2026-04-01 07:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357581%252F93454480_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357581/93454480_hd.jpg not downloaded yet
2026-04-01 07:47:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-a5-sportback-2-0-tfsi-quattro-3352621/details> (referer: None)
2026-04-01 07:47:29 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 1b91743c-5ad9-5357-8f18-cf24c4119a5b with identifier: 3352621
2026-04-01 07:47:29 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 1b91743c-5ad9-5357-8f18-cf24c4119a5b with scrape type 1
2026-04-01 07:47:29 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 1b91743c-5ad9-5357-8f18-cf24c4119a5b
2026-04-01 07:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33527%252F3352621%252F93438789_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33527/3352621/93438789_hd.jpg not downloaded yet
2026-04-01 07:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33527%252F3352621%252F93438790_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33527/3352621/93438790_hd.jpg not downloaded yet
2026-04-01 07:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33527%252F3352621%252F93438791_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33527/3352621/93438791_hd.jpg not downloaded yet
2026-04-01 07:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33527%252F3352621%252F93438792_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33527/3352621/93438792_hd.jpg not downloaded yet
2026-04-01 07:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33527%252F3352621%252F93438793_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33527/3352621/93438793_hd.jpg not downloaded yet
2026-04-01 07:47:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ford-tourneo-courier-1-0-ecoboost-trend-3362683/details> (referer: None)
2026-04-01 07:47:30 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 8b716485-ab6e-5d32-9c9d-ccef14411a23 with identifier: 3362683
2026-04-01 07:47:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 8b716485-ab6e-5d32-9c9d-ccef14411a23 with scrape type 1
2026-04-01 07:47:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 8b716485-ab6e-5d32-9c9d-ccef14411a23
2026-04-01 07:47:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362683%252F93483060_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362683/93483060_hd.jpg not downloaded yet
2026-04-01 07:47:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362683%252F93483051_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362683/93483051_hd.jpg not downloaded yet
2026-04-01 07:47:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362683%252F93483055_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362683/93483055_hd.jpg not downloaded yet
2026-04-01 07:47:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362683%252F93483057_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362683/93483057_hd.jpg not downloaded yet
2026-04-01 07:47:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362683%252F93483058_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362683/93483058_hd.jpg not downloaded yet
2026-04-01 07:47:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/opel-combo-1-3-cdti-ecoflex-selcetion-3362695/details> (referer: None)
2026-04-01 07:47:31 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 0ebd2db3-c506-547d-bc99-ca412f74ef53 with identifier: 3362695
2026-04-01 07:47:31 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 0ebd2db3-c506-547d-bc99-ca412f74ef53 with scrape type 1
2026-04-01 07:47:31 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 0ebd2db3-c506-547d-bc99-ca412f74ef53
2026-04-01 07:47:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362695%252F93483537_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362695/93483537_hd.jpg not downloaded yet
2026-04-01 07:47:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362695%252F93483536_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362695/93483536_hd.jpg not downloaded yet
2026-04-01 07:47:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362695%252F93483545_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362695/93483545_hd.jpg not downloaded yet
2026-04-01 07:47:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362695%252F93483529_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362695/93483529_hd.jpg not downloaded yet
2026-04-01 07:47:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362695%252F93483538_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362695/93483538_hd.jpg not downloaded yet
2026-04-01 07:47:32 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ford-focus-turnier-1-6-ti-vct-fun-x-3362682/details> (referer: None)
2026-04-01 07:47:33 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 855a85be-48ca-5e5f-aefa-4fb8adde72f6 with identifier: 3362682
2026-04-01 07:47:33 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 855a85be-48ca-5e5f-aefa-4fb8adde72f6 with scrape type 1
2026-04-01 07:47:33 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 855a85be-48ca-5e5f-aefa-4fb8adde72f6
2026-04-01 07:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362682%252F93483007_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362682/93483007_hd.jpg not downloaded yet
2026-04-01 07:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362682%252F93483017_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362682/93483017_hd.jpg not downloaded yet
2026-04-01 07:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362682%252F93483022_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362682/93483022_hd.jpg not downloaded yet
2026-04-01 07:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362682%252F93483006_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362682/93483006_hd.jpg not downloaded yet
2026-04-01 07:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362682%252F93483016_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362682/93483016_hd.jpg not downloaded yet
2026-04-01 07:47:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-polo-1-2-tsi-match-3362699/details> (referer: None)
2026-04-01 07:47:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: f4dba5f4-7151-5880-a7ee-db6e6d5702b8 with identifier: 3362699
2026-04-01 07:47:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f4dba5f4-7151-5880-a7ee-db6e6d5702b8 with scrape type 1
2026-04-01 07:47:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item f4dba5f4-7151-5880-a7ee-db6e6d5702b8
2026-04-01 07:47:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362699%252F93483697_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362699/93483697_hd.jpg not downloaded yet
2026-04-01 07:47:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362699%252F93483682_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362699/93483682_hd.jpg not downloaded yet
2026-04-01 07:47:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362699%252F93483689_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362699/93483689_hd.jpg not downloaded yet
2026-04-01 07:47:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362699%252F93483683_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362699/93483683_hd.jpg not downloaded yet
2026-04-01 07:47:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362699%252F93483684_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362699/93483684_hd.jpg not downloaded yet
2026-04-01 07:47:35 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-q3-2-0-tfsi-quattro-3362680/details> (referer: None)
2026-04-01 07:47:35 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: a0bfcc1a-ff00-5010-a4e5-c296e9c4df3b with identifier: 3362680
2026-04-01 07:47:35 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a0bfcc1a-ff00-5010-a4e5-c296e9c4df3b with scrape type 1
2026-04-01 07:47:35 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item a0bfcc1a-ff00-5010-a4e5-c296e9c4df3b
2026-04-01 07:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362680%252F93482921_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362680/93482921_hd.jpg not downloaded yet
2026-04-01 07:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362680%252F93482907_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362680/93482907_hd.jpg not downloaded yet
2026-04-01 07:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362680%252F93482910_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362680/93482910_hd.jpg not downloaded yet
2026-04-01 07:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362680%252F93482908_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362680/93482908_hd.jpg not downloaded yet
2026-04-01 07:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362680%252F93482918_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:47:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362680/93482918_hd.jpg not downloaded yet
2026-04-01 07:48:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 18 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 07:48:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 07:49:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 07:49:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 07:50:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 07:50:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 07:50:16 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 3566bbdb-3e02-578f-a4fb-09d37897e274 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360080/93590988_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:50:18 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 4f0de064-e711-5575-9166-3d18fba5bf5d due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360079/93588689_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:50:19 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 5c5b7dc0-0d5d-5a15-9dda-d2e5b8cc16ce due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33597/3359679/93534563_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:50:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item e2cc5ff5-2036-50e5-a4de-52490ea215b5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360084/93591137_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:50:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item e67f63f0-717d-5212-8025-24cb206fd0ea due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33551/3355036/93607640_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:50:22 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 80ec52c9-52e5-539b-9bc4-562765b36062 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360083/93588745_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:50:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 1cb44775-5daf-5d7a-9fbe-a9ac60c621d2 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360077/93591090_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:50:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item f0da4c24-4ce3-50bc-ae1e-2f038adb2ba6 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362669/93482479_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:51:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 07:51:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 07:52:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 07:52:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 07:53:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 07:53:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 07:53:16 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 2595041c-3131-5750-8b11-40d3e633f77f due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362691/93483366_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:53:18 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 9596939d-952c-5ba8-87f8-831d38662e87 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362676/93482767_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:53:19 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 066652a8-7812-57ae-ab95-81add126dea4 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33576/3357581/93454480_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:53:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 1b91743c-5ad9-5357-8f18-cf24c4119a5b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33527/3352621/93438793_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:53:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 8b716485-ab6e-5d32-9c9d-ccef14411a23 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362683/93483058_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:53:22 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 0ebd2db3-c506-547d-bc99-ca412f74ef53 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362695/93483538_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:53:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 855a85be-48ca-5e5f-aefa-4fb8adde72f6 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362682/93483016_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:53:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item f4dba5f4-7151-5880-a7ee-db6e6d5702b8 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362699/93483684_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:54:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 07:54:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 07:55:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 07:55:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 07:56:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 07:56:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 07:56:16 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item a0bfcc1a-ff00-5010-a4e5-c296e9c4df3b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362680/93482918_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:56:18 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item a0bfcc1a-ff00-5010-a4e5-c296e9c4df3b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362680/93482908_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:56:19 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item a0bfcc1a-ff00-5010-a4e5-c296e9c4df3b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362680/93482910_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:56:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item a0bfcc1a-ff00-5010-a4e5-c296e9c4df3b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362680/93482907_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:56:22 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: a0bfcc1a-ff00-5010-a4e5-c296e9c4df3b with identifier: 3362680
2026-04-01 07:56:22 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 07:56:22 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=white+Metallic HTTP/1.1" 200 None
2026-04-01 07:56:22 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 07:56:22 [urllib3.connectionpool] DEBUG: Resetting dropped connection: alxsourcingstorageprod.table.core.windows.net
2026-04-01 07:56:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 07:56:22 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 07:56:22 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=Petrol HTTP/1.1" 200 None
2026-04-01 07:56:22 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 07:56:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 07:56:22 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: a0bfcc1a-ff00-5010-a4e5-c296e9c4df3b with identifier: 3362680
2026-04-01 07:56:22 [scrapy.core.scraper] INFO: Dropped: Dropping expired listing: a0bfcc1a-ff00-5010-a4e5-c296e9c4df3b with expiration date: 2026-04-01T07:30:00Z
{'auction_title': 'Auction 81210',
 'body_type': 'All-terrain vehicle',
 'category': 'All-terrain vehicle',
 'color': 'white Metallic',
 'currency': 'EUR',
 'emission_standard': 'EURO 5',
 'engine': '2.0L',
 'engine_horse_power': 170,
 'expiration_date': '2026-04-01T07:30:00Z',
 'fuel_type': 'Petrol',
 'id': 'a0bfcc1a-ff00-5010-a4e5-c296e9c4df3b',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'link_to_web_offer': 'https://autobid.de/en/item/audi-q3-2-0-tfsi-quattro-3362680',
 'listing': 'auction',
 'location': 'Braunschweig',
 'make': 'Audi',
 'model': 'Q3',
 'odometer': 165700,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 5,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33627/3362680/93482921_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362680/93482907_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362680/93482910_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362680/93482908_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362680/93482918_hd.jpg'],
 'photos': [],
 'price': 5500,
 'price_includes_vat': False,
 'registration_date': '2013-12-01',
 'seats_number': 5,
 'seller_name': 'Trading firm',
 'title': 'Audi Q3 2,0 TFSI quattro',
 'transmission': 'manual',
 'vin': None,
 'year': 2013}
2026-04-01 07:56:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:56:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362680%27 HTTP/1.1" 200 None
2026-04-01 07:56:22 [autobid.de] INFO: Saving data for 3362680: {'created_time': 1775030182.43197, 'last_price_update_time': 1775030182.431981, 'auction_closing_time': 1775028600.0}
2026-04-01 07:56:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:56:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3362680') HTTP/1.1" 204 0
2026-04-01 07:56:22 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item a0bfcc1a-ff00-5010-a4e5-c296e9c4df3b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362680/93482921_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:56:23 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item f4dba5f4-7151-5880-a7ee-db6e6d5702b8 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362699/93483683_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:56:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item f4dba5f4-7151-5880-a7ee-db6e6d5702b8 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362699/93483689_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:56:26 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item f4dba5f4-7151-5880-a7ee-db6e6d5702b8 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362699/93483682_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:57:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 07:57:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 07:58:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 07:58:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 07:59:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 07:59:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 07:59:16 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: f4dba5f4-7151-5880-a7ee-db6e6d5702b8 with identifier: 3362699
2026-04-01 07:59:16 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 07:59:16 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=grey+Metallic HTTP/1.1" 200 None
2026-04-01 07:59:16 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 07:59:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 07:59:16 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: f4dba5f4-7151-5880-a7ee-db6e6d5702b8 with identifier: 3362699
2026-04-01 07:59:16 [scrapy.core.scraper] INFO: Dropped: Dropping expired listing: f4dba5f4-7151-5880-a7ee-db6e6d5702b8 with expiration date: 2026-04-01T07:30:00Z
{'auction_title': 'Auction 81210',
 'body_type': 'Saloon',
 'category': 'Saloon',
 'color': 'grey Metallic',
 'currency': 'EUR',
 'emission_standard': 'EURO 5',
 'engine': '1.2L',
 'engine_horse_power': 105,
 'expiration_date': '2026-04-01T07:30:00Z',
 'fuel_type': 'Petrol',
 'id': 'f4dba5f4-7151-5880-a7ee-db6e6d5702b8',
 'is_damaged': True,
 'is_operable': True,
 'is_vat_deductible': False,
 'link_to_web_offer': 'https://autobid.de/en/item/vw-polo-1-2-tsi-match-3362699',
 'listing': 'auction',
 'location': 'Braunschweig',
 'make': 'VW',
 'model': 'Polo',
 'odometer': 189200,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 5,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33627/3362699/93483697_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362699/93483682_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362699/93483689_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362699/93483683_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362699/93483684_hd.jpg'],
 'photos': [],
 'price': 2700,
 'price_includes_vat': False,
 'registration_date': '2012-06-01',
 'seats_number': 5,
 'seller_name': 'Subsidiary/authorised dealer of the brand(s): Audi, VW',
 'title': 'VW Polo 1,2 TSI "Match"',
 'transmission': 'manual',
 'vin': None,
 'year': 2012}
2026-04-01 07:59:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:59:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362699%27 HTTP/1.1" 200 None
2026-04-01 07:59:16 [autobid.de] INFO: Saving data for 3362699: {'created_time': 1775030356.768426, 'last_price_update_time': 1775030356.76844, 'auction_closing_time': 1775028600.0}
2026-04-01 07:59:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:59:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3362699') HTTP/1.1" 204 0
2026-04-01 07:59:16 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item f4dba5f4-7151-5880-a7ee-db6e6d5702b8 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362699/93483697_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:59:18 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 855a85be-48ca-5e5f-aefa-4fb8adde72f6 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362682/93483006_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:59:19 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 855a85be-48ca-5e5f-aefa-4fb8adde72f6 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362682/93483022_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:59:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 855a85be-48ca-5e5f-aefa-4fb8adde72f6 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362682/93483017_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:59:22 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 855a85be-48ca-5e5f-aefa-4fb8adde72f6 with identifier: 3362682
2026-04-01 07:59:22 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 07:59:22 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=black+Metallic HTTP/1.1" 200 None
2026-04-01 07:59:23 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 07:59:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 07:59:23 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 855a85be-48ca-5e5f-aefa-4fb8adde72f6 with identifier: 3362682
2026-04-01 07:59:23 [scrapy.core.scraper] INFO: Dropped: Dropping expired listing: 855a85be-48ca-5e5f-aefa-4fb8adde72f6 with expiration date: 2026-04-01T07:30:00Z
{'auction_title': 'Auction 81210',
 'body_type': 'Estate',
 'category': 'Estate',
 'color': 'black Metallic',
 'currency': 'EUR',
 'engine': '1.6L',
 'engine_horse_power': 116,
 'expiration_date': '2026-04-01T07:30:00Z',
 'fuel_type': 'Petrol',
 'id': '855a85be-48ca-5e5f-aefa-4fb8adde72f6',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'link_to_web_offer': 'https://autobid.de/en/item/ford-focus-turnier-1-6-ti-vct-fun-x-3362682',
 'listing': 'auction',
 'location': 'Braunschweig',
 'make': 'Ford',
 'model': 'Focus Turnier',
 'odometer': 107800,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 5,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33627/3362682/93483007_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362682/93483017_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362682/93483022_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362682/93483006_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362682/93483016_hd.jpg'],
 'photos': [],
 'price': 300,
 'price_includes_vat': False,
 'registration_date': '2007-01-01',
 'seats_number': 5,
 'seller_name': 'Subsidiary/authorised dealer of the brand(s): Audi, VW',
 'title': 'Ford Focus Turnier 1,6 Ti-VCT "Fun X"',
 'transmission': 'manual',
 'vin': None,
 'year': 2007}
2026-04-01 07:59:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:59:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362682%27 HTTP/1.1" 200 None
2026-04-01 07:59:23 [autobid.de] INFO: Saving data for 3362682: {'created_time': 1775030363.102267, 'last_price_update_time': 1775030363.102277, 'auction_closing_time': 1775028600.0}
2026-04-01 07:59:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:59:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3362682') HTTP/1.1" 204 0
2026-04-01 07:59:23 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 855a85be-48ca-5e5f-aefa-4fb8adde72f6 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362682/93483007_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:59:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 0ebd2db3-c506-547d-bc99-ca412f74ef53 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362695/93483529_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:59:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 0ebd2db3-c506-547d-bc99-ca412f74ef53 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362695/93483545_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:59:26 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 0ebd2db3-c506-547d-bc99-ca412f74ef53 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362695/93483536_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:00:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 08:00:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 08:01:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 08:01:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 08:02:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 08:02:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 08:02:16 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 0ebd2db3-c506-547d-bc99-ca412f74ef53 with identifier: 3362695
2026-04-01 08:02:16 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 08:02:16 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=red HTTP/1.1" 200 None
2026-04-01 08:02:16 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 08:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 08:02:16 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 08:02:16 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=Diesel HTTP/1.1" 200 None
2026-04-01 08:02:16 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 08:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 08:02:16 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 0ebd2db3-c506-547d-bc99-ca412f74ef53 with identifier: 3362695
2026-04-01 08:02:16 [scrapy.core.scraper] INFO: Dropped: Dropping expired listing: 0ebd2db3-c506-547d-bc99-ca412f74ef53 with expiration date: 2026-04-01T07:30:00Z
{'auction_title': 'Auction 81210',
 'body_type': 'Estate',
 'category': 'Estate',
 'color': 'red',
 'currency': 'EUR',
 'engine': '1.3L',
 'engine_horse_power': 95,
 'expiration_date': '2026-04-01T07:30:00Z',
 'fuel_type': 'Diesel',
 'id': '0ebd2db3-c506-547d-bc99-ca412f74ef53',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'link_to_web_offer': 'https://autobid.de/en/item/opel-combo-1-3-cdti-ecoflex-selcetion-3362695',
 'listing': 'auction',
 'location': 'Göttingen',
 'make': 'Opel',
 'model': 'Combo',
 'odometer': 139400,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 5,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33627/3362695/93483537_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362695/93483536_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362695/93483545_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362695/93483529_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362695/93483538_hd.jpg'],
 'photos': [],
 'price': 3100,
 'price_includes_vat': False,
 'registration_date': '2017-01-01',
 'seats_number': 5,
 'seller_name': 'Subsidiary/authorised dealer of the brand(s): Skoda',
 'title': 'Opel Combo 1,3 CDTI ecoFlex "Selcetion"',
 'transmission': 'manual',
 'vin': None,
 'year': 2017}
2026-04-01 08:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362695%27 HTTP/1.1" 200 None
2026-04-01 08:02:16 [autobid.de] INFO: Saving data for 3362695: {'created_time': 1775030536.919998, 'last_price_update_time': 1775030536.920012, 'auction_closing_time': 1775028600.0}
2026-04-01 08:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:02:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3362695') HTTP/1.1" 204 0
2026-04-01 08:02:16 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 0ebd2db3-c506-547d-bc99-ca412f74ef53 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362695/93483537_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:02:18 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 8b716485-ab6e-5d32-9c9d-ccef14411a23 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362683/93483057_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:02:19 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 8b716485-ab6e-5d32-9c9d-ccef14411a23 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362683/93483055_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:02:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 8b716485-ab6e-5d32-9c9d-ccef14411a23 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362683/93483051_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:02:22 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 8b716485-ab6e-5d32-9c9d-ccef14411a23 with identifier: 3362683
2026-04-01 08:02:22 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 08:02:22 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=white HTTP/1.1" 200 None
2026-04-01 08:02:23 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 08:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 08:02:23 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 8b716485-ab6e-5d32-9c9d-ccef14411a23 with identifier: 3362683
2026-04-01 08:02:23 [scrapy.core.scraper] INFO: Dropped: Dropping expired listing: 8b716485-ab6e-5d32-9c9d-ccef14411a23 with expiration date: 2026-04-01T07:30:00Z
{'auction_title': 'Auction 81210',
 'body_type': 'Van / minibus',
 'category': 'Van / minibus',
 'color': 'white',
 'currency': 'EUR',
 'emission_standard': 'EURO 5',
 'engine': '1.0L',
 'engine_horse_power': 101,
 'expiration_date': '2026-04-01T07:30:00Z',
 'fuel_type': 'Petrol',
 'id': '8b716485-ab6e-5d32-9c9d-ccef14411a23',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'link_to_web_offer': 'https://autobid.de/en/item/ford-tourneo-courier-1-0-ecoboost-trend-3362683',
 'listing': 'auction',
 'location': 'Braunschweig',
 'make': 'Ford',
 'model': 'Tourneo Courier',
 'odometer': 123500,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 5,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33627/3362683/93483060_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362683/93483051_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362683/93483055_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362683/93483057_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362683/93483058_hd.jpg'],
 'photos': [],
 'price': 5900,
 'price_includes_vat': False,
 'registration_date': '2014-12-01',
 'seats_number': 5,
 'seller_name': 'Trading firm',
 'title': 'Ford Tourneo Courier 1,0 EcoBoost "Trend"',
 'transmission': 'manual',
 'vin': None,
 'year': 2014}
2026-04-01 08:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362683%27 HTTP/1.1" 200 None
2026-04-01 08:02:23 [autobid.de] INFO: Saving data for 3362683: {'created_time': 1775030543.076322, 'last_price_update_time': 1775030543.076342, 'auction_closing_time': 1775028600.0}
2026-04-01 08:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3362683') HTTP/1.1" 204 0
2026-04-01 08:02:23 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 8b716485-ab6e-5d32-9c9d-ccef14411a23 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362683/93483060_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:02:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 1b91743c-5ad9-5357-8f18-cf24c4119a5b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33527/3352621/93438792_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:02:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 1b91743c-5ad9-5357-8f18-cf24c4119a5b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33527/3352621/93438791_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:02:26 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 1b91743c-5ad9-5357-8f18-cf24c4119a5b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33527/3352621/93438790_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:03:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 08:03:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 08:04:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 08:04:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 08:05:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 08:05:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 08:05:16 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 1b91743c-5ad9-5357-8f18-cf24c4119a5b with identifier: 3352621
2026-04-01 08:05:16 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 08:05:16 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=grey+Pearl+effect HTTP/1.1" 200 None
2026-04-01 08:05:16 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 08:05:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 08:05:16 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 1b91743c-5ad9-5357-8f18-cf24c4119a5b with identifier: 3352621
2026-04-01 08:05:16 [scrapy.core.scraper] INFO: Dropped: Dropping expired listing: 1b91743c-5ad9-5357-8f18-cf24c4119a5b with expiration date: 2026-04-01T07:30:00Z
{'auction_title': 'Auction 81210',
 'body_type': 'Saloon',
 'category': 'Saloon',
 'color': 'grey Pearl effect',
 'currency': 'EUR',
 'emission_standard': 'EURO 5',
 'engine': '2.0L',
 'engine_horse_power': 211,
 'expiration_date': '2026-04-01T07:30:00Z',
 'fuel_type': 'Petrol',
 'id': '1b91743c-5ad9-5357-8f18-cf24c4119a5b',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'link_to_web_offer': 'https://autobid.de/en/item/audi-a5-sportback-2-0-tfsi-quattro-3352621',
 'listing': 'auction',
 'location': 'Braunschweig',
 'make': 'Audi',
 'model': 'A5 Sportback',
 'odometer': 157200,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 5,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33527/3352621/93438789_hd.jpg',
                'https://cdn.autobid.de/data/cars/33527/3352621/93438790_hd.jpg',
                'https://cdn.autobid.de/data/cars/33527/3352621/93438791_hd.jpg',
                'https://cdn.autobid.de/data/cars/33527/3352621/93438792_hd.jpg',
                'https://cdn.autobid.de/data/cars/33527/3352621/93438793_hd.jpg'],
 'photos': [],
 'price': 6600,
 'price_includes_vat': False,
 'registration_date': '2013-02-01',
 'seats_number': 4,
 'seller_name': 'Subsidiary/authorised dealer of the brand(s): Audi, VW',
 'title': 'Audi A5 Sportback 2,0 TFSI quattro',
 'transmission': 'manual',
 'vin': None,
 'year': 2013}
2026-04-01 08:05:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:05:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352621%27 HTTP/1.1" 200 None
2026-04-01 08:05:16 [autobid.de] INFO: Saving data for 3352621: {'created_time': 1775030716.72514, 'last_price_update_time': 1775030716.725154, 'auction_closing_time': 1775028600.0}
2026-04-01 08:05:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:05:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3352621') HTTP/1.1" 204 0
2026-04-01 08:05:16 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 1b91743c-5ad9-5357-8f18-cf24c4119a5b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33527/3352621/93438789_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:05:18 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 066652a8-7812-57ae-ab95-81add126dea4 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33576/3357581/93454479_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:05:19 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 066652a8-7812-57ae-ab95-81add126dea4 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33576/3357581/93454478_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:05:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 066652a8-7812-57ae-ab95-81add126dea4 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33576/3357581/93454477_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:05:22 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 066652a8-7812-57ae-ab95-81add126dea4 with identifier: 3357581
2026-04-01 08:05:22 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 08:05:23 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=silver+Metallic HTTP/1.1" 200 None
2026-04-01 08:05:23 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 08:05:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 08:05:23 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 066652a8-7812-57ae-ab95-81add126dea4 with identifier: 3357581
2026-04-01 08:05:23 [scrapy.core.scraper] INFO: Dropped: Dropping expired listing: 066652a8-7812-57ae-ab95-81add126dea4 with expiration date: 2026-04-01T07:30:00Z
{'auction_title': 'Auction 81210',
 'body_type': 'Van / minibus',
 'category': 'Van / minibus',
 'color': 'silver Metallic',
 'currency': 'EUR',
 'emission_standard': 'EURO 6',
 'engine': '1.6L',
 'engine_horse_power': 116,
 'expiration_date': '2026-04-01T07:30:00Z',
 'fuel_type': 'Diesel',
 'id': '066652a8-7812-57ae-ab95-81add126dea4',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'link_to_web_offer': 'https://autobid.de/en/item/vw-golf-sportsvan-1-6-tdi-bmt-dsg-comfortline-3357581',
 'listing': 'auction',
 'location': 'Braunschweig',
 'make': 'VW',
 'model': 'Golf Sportsvan',
 'odometer': 43100,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 5,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33576/3357581/93454476_hd.jpg',
                'https://cdn.autobid.de/data/cars/33576/3357581/93454477_hd.jpg',
                'https://cdn.autobid.de/data/cars/33576/3357581/93454478_hd.jpg',
                'https://cdn.autobid.de/data/cars/33576/3357581/93454479_hd.jpg',
                'https://cdn.autobid.de/data/cars/33576/3357581/93454480_hd.jpg'],
 'photos': [],
 'price': 12200,
 'price_includes_vat': False,
 'registration_date': '2017-04-01',
 'seats_number': 5,
 'seller_name': 'Subsidiary/authorised dealer of the brand(s): VW',
 'title': 'VW Golf Sportsvan 1,6 TDI BMT DSG "Comfortline"',
 'transmission': 'automatic',
 'vin': None,
 'year': 2017}
2026-04-01 08:05:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:05:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357581%27 HTTP/1.1" 200 None
2026-04-01 08:05:23 [autobid.de] INFO: Saving data for 3357581: {'created_time': 1775030723.099279, 'last_price_update_time': 1775030723.099299, 'auction_closing_time': 1775028600.0}
2026-04-01 08:05:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:05:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3357581') HTTP/1.1" 204 0
2026-04-01 08:05:23 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 066652a8-7812-57ae-ab95-81add126dea4 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33576/3357581/93454476_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:05:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 9596939d-952c-5ba8-87f8-831d38662e87 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362676/93482755_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:05:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 9596939d-952c-5ba8-87f8-831d38662e87 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362676/93482768_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:05:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 9596939d-952c-5ba8-87f8-831d38662e87 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362676/93482756_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:06:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 08:06:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 08:07:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 08:07:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 08:08:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 08:08:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 08:08:16 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 9596939d-952c-5ba8-87f8-831d38662e87 with identifier: 3362676
2026-04-01 08:08:16 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 08:08:16 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=black+Pearl+effect HTTP/1.1" 200 None
2026-04-01 08:08:16 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 08:08:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 08:08:16 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 9596939d-952c-5ba8-87f8-831d38662e87 with identifier: 3362676
2026-04-01 08:08:16 [scrapy.core.scraper] INFO: Dropped: Dropping expired listing: 9596939d-952c-5ba8-87f8-831d38662e87 with expiration date: 2026-04-01T07:30:00Z
{'auction_title': 'Auction 81210',
 'body_type': 'All-terrain vehicle',
 'category': 'All-terrain vehicle',
 'color': 'black Pearl effect',
 'currency': 'EUR',
 'emission_standard': 'EURO 6d-Temp',
 'engine': '2.0L',
 'engine_horse_power': 150,
 'expiration_date': '2026-04-01T07:30:00Z',
 'fuel_type': 'Diesel',
 'id': '9596939d-952c-5ba8-87f8-831d38662e87',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_included': True,
 'link_to_web_offer': 'https://autobid.de/en/item/skoda-kodiaq-2-0-tdi-4x4-style-3362676',
 'listing': 'auction',
 'location': 'Siek',
 'make': 'Skoda',
 'model': 'Kodiaq',
 'odometer': 208000,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 5,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33627/3362676/93482769_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362676/93482756_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362676/93482768_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362676/93482755_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362676/93482767_hd.jpg'],
 'photos': [],
 'price': 13000,
 'price_includes_vat': True,
 'registration_date': '2019-11-01',
 'seats_number': 5,
 'seller_name': 'Fleet company',
 'title': 'Skoda Kodiaq 2,0 TDI 4x4 "Style"',
 'transmission': 'manual',
 'vin': None,
 'year': 2019}
2026-04-01 08:08:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:08:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362676%27 HTTP/1.1" 200 None
2026-04-01 08:08:16 [autobid.de] INFO: Saving data for 3362676: {'created_time': 1775030896.753523, 'last_price_update_time': 1775030896.753536, 'auction_closing_time': 1775028600.0}
2026-04-01 08:08:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:08:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3362676') HTTP/1.1" 204 0
2026-04-01 08:08:16 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 9596939d-952c-5ba8-87f8-831d38662e87 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362676/93482769_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:08:18 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 2595041c-3131-5750-8b11-40d3e633f77f due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362691/93483367_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:08:19 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 2595041c-3131-5750-8b11-40d3e633f77f due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362691/93483360_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:08:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 2595041c-3131-5750-8b11-40d3e633f77f due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362691/93483355_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:08:22 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 2595041c-3131-5750-8b11-40d3e633f77f with identifier: 3362691
2026-04-01 08:08:22 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 2595041c-3131-5750-8b11-40d3e633f77f with identifier: 3362691
2026-04-01 08:08:22 [scrapy.core.scraper] INFO: Dropped: Dropping expired listing: 2595041c-3131-5750-8b11-40d3e633f77f with expiration date: 2026-04-01T07:30:00Z
{'auction_title': 'Auction 81210',
 'body_type': 'All-terrain vehicle',
 'category': 'All-terrain vehicle',
 'color': 'black Pearl effect',
 'currency': 'EUR',
 'emission_standard': 'EURO 6d-Temp',
 'engine': '2.0L',
 'engine_horse_power': 150,
 'expiration_date': '2026-04-01T07:30:00Z',
 'fuel_type': 'Diesel',
 'id': '2595041c-3131-5750-8b11-40d3e633f77f',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_included': True,
 'link_to_web_offer': 'https://autobid.de/en/item/skoda-karoq-2-0-tdi-4x4-dsg-style-3362691',
 'listing': 'auction',
 'location': 'Siek',
 'make': 'Skoda',
 'model': 'Karoq',
 'odometer': 149300,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 5,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33627/3362691/93483354_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362691/93483355_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362691/93483360_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362691/93483367_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362691/93483366_hd.jpg'],
 'photos': [],
 'price': 15100,
 'price_includes_vat': True,
 'registration_date': '2019-06-01',
 'seats_number': 5,
 'seller_name': 'Fleet company',
 'title': 'Skoda Karoq 2,0 TDI 4x4 DSG "Style"',
 'transmission': 'automatic',
 'vin': None,
 'year': 2019}
2026-04-01 08:08:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:08:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362691%27 HTTP/1.1" 200 None
2026-04-01 08:08:22 [autobid.de] INFO: Saving data for 3362691: {'created_time': 1775030902.926799, 'last_price_update_time': 1775030902.926816, 'auction_closing_time': 1775028600.0}
2026-04-01 08:08:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:08:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3362691') HTTP/1.1" 204 0
2026-04-01 08:08:22 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 2595041c-3131-5750-8b11-40d3e633f77f due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362691/93483354_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:08:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item f0da4c24-4ce3-50bc-ae1e-2f038adb2ba6 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362669/93482475_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:08:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item f0da4c24-4ce3-50bc-ae1e-2f038adb2ba6 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362669/93482467_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:08:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item f0da4c24-4ce3-50bc-ae1e-2f038adb2ba6 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362669/93482457_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:09:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 08:09:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 08:10:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 08:10:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 08:11:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 08:11:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 08:11:16 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: f0da4c24-4ce3-50bc-ae1e-2f038adb2ba6 with identifier: 3362669
2026-04-01 08:11:16 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: f0da4c24-4ce3-50bc-ae1e-2f038adb2ba6 with identifier: 3362669
2026-04-01 08:11:16 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: f0da4c24-4ce3-50bc-ae1e-2f038adb2ba6 with identifier: 3362669 to the API
2026-04-01 08:11:16 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 08:11:16 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 08:11:16 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362669/93482459_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 81210',
 'body_type': 'Estate',
 'category': 'Estate',
 'color': 'white Metallic',
 'currency': 'EUR',
 'emission_standard': 'EURO 6',
 'engine': '2.0L',
 'engine_horse_power': 252,
 'fuel_type': 'Petrol',
 'id': 'f0da4c24-4ce3-50bc-ae1e-2f038adb2ba6',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'link_to_web_offer': 'https://autobid.de/en/item/audi-a6-avant-2-0-tfsi-quattro-s-tronic-3362669',
 'listing': 'auction',
 'location': 'Göttingen',
 'make': 'Audi',
 'model': 'A6 Avant',
 'odometer': 100900,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 5,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33627/3362669/93482459_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362669/93482457_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362669/93482467_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362669/93482475_hd.jpg',
                'https://cdn.autobid.de/data/cars/33627/3362669/93482479_hd.jpg'],
 'photos': [],
 'price': 18000,
 'price_includes_vat': False,
 'registration_date': '2018-12-01',
 'seats_number': 5,
 'seller_name': None,
 'title': 'Audi A6 Avant 2,0 TFSI quattro S-tronic',
 'transmission': 'automatic',
 'vin': None,
 'year': 2018}
2026-04-01 08:11:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:11:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362669%27 HTTP/1.1" 200 None
2026-04-01 08:11:17 [autobid.de] INFO: Saving data for 3362669: {'created_time': 1775031077.017338, 'last_price_update_time': 1775031077.017352}
2026-04-01 08:11:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:11:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3362669') HTTP/1.1" 204 0
2026-04-01 08:11:17 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item f0da4c24-4ce3-50bc-ae1e-2f038adb2ba6 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362669/93482459_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:11:18 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 1cb44775-5daf-5d7a-9fbe-a9ac60c621d2 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360077/93591089_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:11:19 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 1cb44775-5daf-5d7a-9fbe-a9ac60c621d2 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360077/93591088_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:11:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 1cb44775-5daf-5d7a-9fbe-a9ac60c621d2 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360077/93591087_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:11:22 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 1cb44775-5daf-5d7a-9fbe-a9ac60c621d2 with identifier: 3360077
2026-04-01 08:11:22 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 08:11:22 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=blue+Metallic HTTP/1.1" 200 None
2026-04-01 08:11:23 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 08:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 08:11:23 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 1cb44775-5daf-5d7a-9fbe-a9ac60c621d2 with identifier: 3360077
2026-04-01 08:11:23 [scrapy.core.scraper] INFO: Dropped: Dropping expired listing: 1cb44775-5daf-5d7a-9fbe-a9ac60c621d2 with expiration date: 2026-04-01T07:30:00Z
{'auction_title': 'Auction 81241',
 'body_type': 'Saloon',
 'category': 'Saloon',
 'color': 'blue Metallic',
 'currency': 'EUR',
 'emission_standard': 'EURO 5',
 'engine': '1.2L',
 'engine_horse_power': 105,
 'expiration_date': '2026-04-01T07:30:00Z',
 'fuel_type': 'Petrol',
 'id': '1cb44775-5daf-5d7a-9fbe-a9ac60c621d2',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'link_to_web_offer': 'https://autobid.de/en/item/skoda-fabia-1-2-tsi-dsg-family-3360077',
 'listing': 'auction',
 'location': 'Lübeck',
 'make': 'Skoda',
 'model': 'Fabia',
 'odometer': 300600,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 5,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33601/3360077/93591086_hd.jpg',
                'https://cdn.autobid.de/data/cars/33601/3360077/93591087_hd.jpg',
                'https://cdn.autobid.de/data/cars/33601/3360077/93591088_hd.jpg',
                'https://cdn.autobid.de/data/cars/33601/3360077/93591089_hd.jpg',
                'https://cdn.autobid.de/data/cars/33601/3360077/93591090_hd.jpg'],
 'photos': [],
 'price': 800,
 'price_includes_vat': False,
 'registration_date': '2011-06-01',
 'seats_number': 5,
 'seller_name': 'Senger Holstein GmbH',
 'title': 'Skoda Fabia 1,2 TSI DSG "Family"',
 'transmission': 'automatic',
 'vin': None,
 'year': 2011}
2026-04-01 08:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360077%27 HTTP/1.1" 200 None
2026-04-01 08:11:23 [autobid.de] INFO: Saving data for 3360077: {'created_time': 1775031083.111828, 'last_price_update_time': 1775031083.111839, 'auction_closing_time': 1775028600.0}
2026-04-01 08:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3360077') HTTP/1.1" 204 0
2026-04-01 08:11:23 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 1cb44775-5daf-5d7a-9fbe-a9ac60c621d2 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360077/93591086_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:11:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 80ec52c9-52e5-539b-9bc4-562765b36062 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360083/93588744_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:11:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 80ec52c9-52e5-539b-9bc4-562765b36062 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360083/93588743_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:11:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 80ec52c9-52e5-539b-9bc4-562765b36062 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360083/93588742_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:12:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 1 items (at 1 items/min)
2026-04-01 08:12:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 08:13:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 1 items (at 0 items/min)
2026-04-01 08:13:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 08:14:07 [scrapy.extensions.logstats] INFO: Crawled 18 pages (at 0 pages/min), scraped 1 items (at 0 items/min)
2026-04-01 08:14:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 180MiB
2026-04-01 08:14:16 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 80ec52c9-52e5-539b-9bc4-562765b36062 with identifier: 3360083
2026-04-01 08:14:16 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 80ec52c9-52e5-539b-9bc4-562765b36062 with identifier: 3360083
2026-04-01 08:14:16 [scrapy.core.scraper] INFO: Dropped: Dropping expired listing: 80ec52c9-52e5-539b-9bc4-562765b36062 with expiration date: 2026-04-01T07:30:00Z
{'auction_title': 'Auction 81241',
 'body_type': 'Convertible / Roadster',
 'category': 'Convertible / Roadster',
 'color': 'grey Pearl effect',
 'currency': 'EUR',
 'emission_standard': 'EURO 6',
 'engine': '1.8L',
 'engine_horse_power': 170,
 'expiration_date': '2026-04-01T07:30:00Z',
 'fuel_type': 'Petrol',
 'id': '80ec52c9-52e5-539b-9bc4-562765b36062',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'link_to_web_offer': 'https://autobid.de/en/item/audi-a5-cabriolet-1-8-tfsi-multitronic-s-line-3360083',
 'listing': 'auction',
 'location': 'Lübeck',
 'make': 'Audi',
 'model': 'A5 Cabriolet',
 'odometer': 134100,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 5,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33601/3360083/93588741_hd.jpg',
                'https://cdn.autobid.de/data/cars/33601/3360083/93588742_hd.jpg',
                'https://cdn.autobid.de/data/cars/33601/3360083/93588743_hd.jpg',
                'https://cdn.autobid.de/data/cars/33601/3360083/93588744_hd.jpg',
                'https://cdn.autobid.de/data/cars/33601/3360083/93588745_hd.jpg'],
 'photos': [],
 'price': 6500,
 'price_includes_vat': False,
 'registration_date': '2013-06-01',
 'seats_number': 4,
 'seller_name': 'Senger Automobile GmbH',
 'title': 'Audi A5 Cabriolet 1,8 TFSI Multitronic "S-line"',
 'transmission': 'automatic',
 'vin': None,
 'year': 2013}
2026-04-01 08:14:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360083%27 HTTP/1.1" 200 None
2026-04-01 08:14:16 [autobid.de] INFO: Saving data for 3360083: {'created_time': 1775031256.617592, 'last_price_update_time': 1775031256.617605, 'auction_closing_time': 1775028600.0}
2026-04-01 08:14:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3360083') HTTP/1.1" 204 0
2026-04-01 08:14:16 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 80ec52c9-52e5-539b-9bc4-562765b36062 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360083/93588741_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:14:18 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item e67f63f0-717d-5212-8025-24cb206fd0ea due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33551/3355036/93607643_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:14:19 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item e67f63f0-717d-5212-8025-24cb206fd0ea due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33551/3355036/93607647_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:14:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item e67f63f0-717d-5212-8025-24cb206fd0ea due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33551/3355036/93607639_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:14:22 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: e67f63f0-717d-5212-8025-24cb206fd0ea with identifier: 3355036
2026-04-01 08:14:22 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 08:14:22 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=green+Pearl+effect HTTP/1.1" 200 None
2026-04-01 08:14:23 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 08:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 08:14:23 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: e67f63f0-717d-5212-8025-24cb206fd0ea with identifier: 3355036
2026-04-01 08:14:23 [scrapy.core.scraper] INFO: Dropped: Dropping expired listing: e67f63f0-717d-5212-8025-24cb206fd0ea with expiration date: 2026-04-01T07:30:00Z
{'auction_title': 'Auction 81241',
 'body_type': 'All-terrain vehicle',
 'category': 'All-terrain vehicle',
 'color': 'green Pearl effect',
 'currency': 'EUR',
 'emission_standard': 'EURO 5',
 'engine': '2.0L',
 'engine_horse_power': 140,
 'expiration_date': '2026-04-01T07:30:00Z',
 'fuel_type': 'Diesel',
 'id': 'e67f63f0-717d-5212-8025-24cb206fd0ea',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'link_to_web_offer': 'https://autobid.de/en/item/vw-tiguan-2-0-tdi-bmt-team-3355036',
 'listing': 'auction',
 'location': 'Oelde-Stromberg',
 'make': 'VW',
 'model': 'Tiguan',
 'odometer': 160900,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 5,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33551/3355036/93607629_hd.jpg',
                'https://cdn.autobid.de/data/cars/33551/3355036/93607639_hd.jpg',
                'https://cdn.autobid.de/data/cars/33551/3355036/93607647_hd.jpg',
                'https://cdn.autobid.de/data/cars/33551/3355036/93607643_hd.jpg',
                'https://cdn.autobid.de/data/cars/33551/3355036/93607640_hd.jpg'],
 'photos': [],
 'price': 4450,
 'price_includes_vat': False,
 'registration_date': '2011-06-01',
 'seats_number': 5,
 'seller_name': 'Senger Holstein GmbH',
 'title': 'VW Tiguan 2,0 TDI BMT "Team"',
 'transmission': 'manual',
 'vin': None,
 'year': 2011}
2026-04-01 08:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355036%27 HTTP/1.1" 200 None
2026-04-01 08:14:23 [autobid.de] INFO: Saving data for 3355036: {'created_time': 1775031263.061305, 'last_price_update_time': 1775031263.061317, 'auction_closing_time': 1775028600.0}
2026-04-01 08:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3355036') HTTP/1.1" 204 0
2026-04-01 08:14:23 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item e67f63f0-717d-5212-8025-24cb206fd0ea due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33551/3355036/93607629_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:14:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item e2cc5ff5-2036-50e5-a4de-52490ea215b5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360084/93591136_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:14:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item e2cc5ff5-2036-50e5-a4de-52490ea215b5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360084/93591134_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:14:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/seat-ateca-1-5-tsi-act-xcellence-3362931/details> (referer: None)
2026-04-01 08:14:26 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 976070b1-c53a-5f93-be5c-6206108a7858 with identifier: 3362931
2026-04-01 08:14:26 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 976070b1-c53a-5f93-be5c-6206108a7858 with scrape type 1
2026-04-01 08:14:26 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 976070b1-c53a-5f93-be5c-6206108a7858
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33630%252F3362931%252F93586469_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33630/3362931/93586469_hd.jpg not downloaded yet
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33630%252F3362931%252F93586470_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33630/3362931/93586470_hd.jpg not downloaded yet
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33630%252F3362931%252F93586471_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33630/3362931/93586471_hd.jpg not downloaded yet
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33630%252F3362931%252F93586472_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33630/3362931/93586472_hd.jpg not downloaded yet
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33630%252F3362931%252F93586473_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33630/3362931/93586473_hd.jpg not downloaded yet
2026-04-01 08:14:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=2> (referer: None)
2026-04-01 08:14:26 [autobid.de] INFO: Page 2/141: found 20 items
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366099%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [autobid.de] INFO: Scrape type for 3366099: 1
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366092%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [autobid.de] INFO: Scrape type for 3366092: 1
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366094%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [autobid.de] INFO: Scrape type for 3366094: 1
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366091%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [autobid.de] INFO: Scrape type for 3366091: 1
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366093%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [autobid.de] INFO: Scrape type for 3366093: 1
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365151%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [autobid.de] INFO: Scrape type for 3365151: 1
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365064%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [autobid.de] INFO: Scrape type for 3365064: 1
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366338%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [autobid.de] INFO: Scrape type for 3366338: 1
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366340%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [autobid.de] INFO: Scrape type for 3366340: 1
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364756%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [autobid.de] INFO: Scrape type for 3364756: 1
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364754%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [autobid.de] INFO: Scrape type for 3364754: 1
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365542%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [autobid.de] INFO: Scrape type for 3365542: 1
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365047%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [autobid.de] INFO: Scrape type for 3365047: 1
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364755%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [autobid.de] INFO: Scrape type for 3364755: 1
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365251%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [autobid.de] INFO: Scrape type for 3365251: 1
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366339%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [autobid.de] INFO: Scrape type for 3366339: 1
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365156%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [autobid.de] INFO: Scrape type for 3365156: 1
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365544%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [autobid.de] INFO: Scrape type for 3365544: 1
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364604%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [autobid.de] INFO: Scrape type for 3364604: 1
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357570%27 HTTP/1.1" 200 None
2026-04-01 08:14:26 [autobid.de] INFO: Scrape type for 3357570: 1
2026-04-01 08:14:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item e2cc5ff5-2036-50e5-a4de-52490ea215b5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360084/93591133_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:14:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-golf-variant-1-6-tdi-bluemotion-trendline-3362684/details> (referer: None)
2026-04-01 08:14:27 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 2f8544d1-65bf-566a-8914-8d7fe64bd511 with identifier: 3362684
2026-04-01 08:14:27 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 2f8544d1-65bf-566a-8914-8d7fe64bd511 with scrape type 1
2026-04-01 08:14:27 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 2f8544d1-65bf-566a-8914-8d7fe64bd511
2026-04-01 08:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362684%252F93483094_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:14:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362684/93483094_hd.jpg not downloaded yet
2026-04-01 08:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362684%252F93483089_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:14:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362684/93483089_hd.jpg not downloaded yet
2026-04-01 08:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362684%252F93483098_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:14:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362684/93483098_hd.jpg not downloaded yet
2026-04-01 08:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362684%252F93483083_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:14:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362684/93483083_hd.jpg not downloaded yet
2026-04-01 08:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362684%252F93483104_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:14:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362684/93483104_hd.jpg not downloaded yet
2026-04-01 08:14:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-amarok-doppelcabine-3-0-tdi-bmt-4motion-tiptronic-comfortline-3357570/details> (referer: None)
2026-04-01 08:14:30 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 07dab192-7e58-593b-9e59-cc2c85c84481 with identifier: 3357570
2026-04-01 08:14:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 07dab192-7e58-593b-9e59-cc2c85c84481 with scrape type 1
2026-04-01 08:14:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 07dab192-7e58-593b-9e59-cc2c85c84481
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357570%252F93575107_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357570/93575107_hd.jpg not downloaded yet
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357570%252F93575108_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357570/93575108_hd.jpg not downloaded yet
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357570%252F93575109_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357570/93575109_hd.jpg not downloaded yet
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357570%252F93575110_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357570/93575110_hd.jpg not downloaded yet
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357570%252F93575111_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357570/93575111_hd.jpg not downloaded yet
2026-04-01 08:14:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=3> (referer: None)
2026-04-01 08:14:30 [autobid.de] INFO: Page 3/141: found 20 items
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365543%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [autobid.de] INFO: Scrape type for 3365543: 1
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357571%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [autobid.de] INFO: Scrape type for 3357571: 1
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273349860%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [autobid.de] INFO: Scrape type for 3349860: 1
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355071%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [autobid.de] INFO: Scrape type for 3355071: 1
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355810%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [autobid.de] INFO: Scrape type for 3355810: 1
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357023%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [autobid.de] INFO: Scrape type for 3357023: 1
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366330%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [autobid.de] INFO: Scrape type for 3366330: 1
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366331%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [autobid.de] INFO: Scrape type for 3366331: 1
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355072%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [autobid.de] INFO: Scrape type for 3355072: 1
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364608%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [autobid.de] INFO: Scrape type for 3364608: 1
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364609%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [autobid.de] INFO: Scrape type for 3364609: 1
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366328%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [autobid.de] INFO: Scrape type for 3366328: 1
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366332%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [autobid.de] INFO: Scrape type for 3366332: 1
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366333%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [autobid.de] INFO: Scrape type for 3366333: 1
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366335%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [autobid.de] INFO: Scrape type for 3366335: 1
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366336%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [autobid.de] INFO: Scrape type for 3366336: 1
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366341%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [autobid.de] INFO: Scrape type for 3366341: 1
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366369%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [autobid.de] INFO: Scrape type for 3366369: 1
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366376%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [autobid.de] INFO: Scrape type for 3366376: 1
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366371%27 HTTP/1.1" 200 None
2026-04-01 08:14:30 [autobid.de] INFO: Scrape type for 3366371: 1
2026-04-01 08:15:07 [scrapy.extensions.logstats] INFO: Crawled 23 pages (at 5 pages/min), scraped 1 items (at 0 items/min)
2026-04-01 08:15:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 189MiB
2026-04-01 08:16:07 [scrapy.extensions.logstats] INFO: Crawled 23 pages (at 0 pages/min), scraped 1 items (at 0 items/min)
2026-04-01 08:16:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 190MiB
2026-04-01 08:17:07 [scrapy.extensions.logstats] INFO: Crawled 23 pages (at 0 pages/min), scraped 1 items (at 0 items/min)
2026-04-01 08:17:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 192MiB
2026-04-01 08:17:16 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: e2cc5ff5-2036-50e5-a4de-52490ea215b5 with identifier: 3360084
2026-04-01 08:17:16 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 08:17:16 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=red+Metallic HTTP/1.1" 200 None
2026-04-01 08:17:16 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 08:17:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 08:17:16 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: e2cc5ff5-2036-50e5-a4de-52490ea215b5 with identifier: 3360084
2026-04-01 08:17:16 [scrapy.core.scraper] INFO: Dropped: Dropping expired listing: e2cc5ff5-2036-50e5-a4de-52490ea215b5 with expiration date: 2026-04-01T07:30:00Z
{'auction_title': 'Auction 81241',
 'body_type': 'Saloon',
 'category': 'Saloon',
 'color': 'red Metallic',
 'currency': 'EUR',
 'emission_standard': 'EURO 6',
 'engine': '2.0L',
 'engine_horse_power': 180,
 'expiration_date': '2026-04-01T07:30:00Z',
 'fuel_type': 'Diesel',
 'id': 'e2cc5ff5-2036-50e5-a4de-52490ea215b5',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'link_to_web_offer': 'https://autobid.de/en/item/ford-mondeo-2-0-tdci-powershift-titanium-3360084',
 'listing': 'auction',
 'location': 'Lübeck',
 'make': 'Ford',
 'model': 'Mondeo',
 'odometer': 139300,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 5,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33601/3360084/93591132_hd.jpg',
                'https://cdn.autobid.de/data/cars/33601/3360084/93591133_hd.jpg',
                'https://cdn.autobid.de/data/cars/33601/3360084/93591134_hd.jpg',
                'https://cdn.autobid.de/data/cars/33601/3360084/93591136_hd.jpg',
                'https://cdn.autobid.de/data/cars/33601/3360084/93591137_hd.jpg'],
 'photos': [],
 'price': 7700,
 'price_includes_vat': False,
 'registration_date': '2015-06-01',
 'seats_number': 5,
 'seller_name': 'Senger Automobile GmbH',
 'title': 'Ford Mondeo 2,0 TDCI Powershift "Titanium"',
 'transmission': 'automatic',
 'vin': None,
 'year': 2015}
2026-04-01 08:17:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360084%27 HTTP/1.1" 200 None
2026-04-01 08:17:16 [autobid.de] INFO: Saving data for 3360084: {'created_time': 1775031436.757093, 'last_price_update_time': 1775031436.757106, 'auction_closing_time': 1775028600.0}
2026-04-01 08:17:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3360084') HTTP/1.1" 204 0
2026-04-01 08:17:17 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item e2cc5ff5-2036-50e5-a4de-52490ea215b5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360084/93591132_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:17:18 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 5c5b7dc0-0d5d-5a15-9dda-d2e5b8cc16ce due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33597/3359679/93534569_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:17:18 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-t6-1-transporter-kasten-lr-2-0-tdi-sortimo-8-fach-3366371/details> (referer: None)
2026-04-01 08:17:18 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: b368875c-8f77-5680-9087-a40250631e69 with identifier: 3366371
2026-04-01 08:17:18 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b368875c-8f77-5680-9087-a40250631e69 with scrape type 1
2026-04-01 08:17:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 2 photos for item b368875c-8f77-5680-9087-a40250631e69
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33664%252F3366371%252F93614070_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:17:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33664/3366371/93614070_hd.jpg not downloaded yet
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33664%252F3366371%252F93614075_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:17:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33664/3366371/93614075_hd.jpg not downloaded yet
2026-04-01 08:17:18 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=4> (referer: None)
2026-04-01 08:17:18 [autobid.de] INFO: Page 4/141: found 20 items
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366344%27 HTTP/1.1" 200 None
2026-04-01 08:17:18 [autobid.de] INFO: Scrape type for 3366344: 1
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366346%27 HTTP/1.1" 200 None
2026-04-01 08:17:18 [autobid.de] INFO: Scrape type for 3366346: 1
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366370%27 HTTP/1.1" 200 None
2026-04-01 08:17:18 [autobid.de] INFO: Scrape type for 3366370: 1
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366345%27 HTTP/1.1" 200 None
2026-04-01 08:17:18 [autobid.de] INFO: Scrape type for 3366345: 1
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366347%27 HTTP/1.1" 200 None
2026-04-01 08:17:18 [autobid.de] INFO: Scrape type for 3366347: 1
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358299%27 HTTP/1.1" 200 None
2026-04-01 08:17:18 [autobid.de] INFO: Scrape type for 3358299: 1
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359755%27 HTTP/1.1" 200 None
2026-04-01 08:17:18 [autobid.de] INFO: Scrape type for 3359755: 1
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360056%27 HTTP/1.1" 200 None
2026-04-01 08:17:18 [autobid.de] INFO: Scrape type for 3360056: 1
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355058%27 HTTP/1.1" 200 None
2026-04-01 08:17:18 [autobid.de] INFO: Scrape type for 3355058: 1
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357000%27 HTTP/1.1" 200 None
2026-04-01 08:17:18 [autobid.de] INFO: Scrape type for 3357000: 1
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356997%27 HTTP/1.1" 200 None
2026-04-01 08:17:18 [autobid.de] INFO: Scrape type for 3356997: 1
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356998%27 HTTP/1.1" 200 None
2026-04-01 08:17:18 [autobid.de] INFO: Scrape type for 3356998: 1
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356999%27 HTTP/1.1" 200 None
2026-04-01 08:17:18 [autobid.de] INFO: Scrape type for 3356999: 1
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357001%27 HTTP/1.1" 200 None
2026-04-01 08:17:18 [autobid.de] INFO: Scrape type for 3357001: 1
2026-04-01 08:17:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357353%27 HTTP/1.1" 200 None
2026-04-01 08:17:19 [autobid.de] INFO: Scrape type for 3357353: 1
2026-04-01 08:17:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361285%27 HTTP/1.1" 200 None
2026-04-01 08:17:19 [autobid.de] INFO: Scrape type for 3361285: 1
2026-04-01 08:17:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361286%27 HTTP/1.1" 200 None
2026-04-01 08:17:19 [autobid.de] INFO: Scrape type for 3361286: 1
2026-04-01 08:17:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361287%27 HTTP/1.1" 200 None
2026-04-01 08:17:19 [autobid.de] INFO: Scrape type for 3361287: 1
2026-04-01 08:17:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361288%27 HTTP/1.1" 200 None
2026-04-01 08:17:19 [autobid.de] INFO: Scrape type for 3361288: 1
2026-04-01 08:17:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361289%27 HTTP/1.1" 200 None
2026-04-01 08:17:19 [autobid.de] INFO: Scrape type for 3361289: 1
2026-04-01 08:17:19 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 5c5b7dc0-0d5d-5a15-9dda-d2e5b8cc16ce due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33597/3359679/93534576_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:17:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-touran-comfortline-1-4-tsi-acc-ahk-front-16-3366376/details> (referer: None)
2026-04-01 08:17:19 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: da10067d-0f0f-5425-845f-788747dea6b3 with identifier: 3366376
2026-04-01 08:17:19 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item da10067d-0f0f-5425-845f-788747dea6b3 with scrape type 1
2026-04-01 08:17:19 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 2 photos for item da10067d-0f0f-5425-845f-788747dea6b3
2026-04-01 08:17:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33664%252F3366376%252F93616550_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:17:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33664/3366376/93616550_hd.jpg not downloaded yet
2026-04-01 08:17:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33664%252F3366376%252F93616544_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:17:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33664/3366376/93616544_hd.jpg not downloaded yet
2026-04-01 08:17:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 5c5b7dc0-0d5d-5a15-9dda-d2e5b8cc16ce due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33597/3359679/93534562_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:17:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/nissan-nv200-1-4-3361289/details> (referer: None)
2026-04-01 08:17:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=5> (referer: None)
2026-04-01 08:17:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 49e79d41-e5b4-5f01-9788-72713b84a85d with identifier: 3361289
2026-04-01 08:17:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 49e79d41-e5b4-5f01-9788-72713b84a85d with scrape type 1
2026-04-01 08:17:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 49e79d41-e5b4-5f01-9788-72713b84a85d
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33613%252F3361289%252F93420687_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:17:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33613/3361289/93420687_hd.jpg not downloaded yet
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33613%252F3361289%252F93420690_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:17:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33613/3361289/93420690_hd.jpg not downloaded yet
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33613%252F3361289%252F93420698_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:17:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33613/3361289/93420698_hd.jpg not downloaded yet
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33613%252F3361289%252F93420692_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:17:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33613/3361289/93420692_hd.jpg not downloaded yet
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33613%252F3361289%252F93420683_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:17:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33613/3361289/93420683_hd.jpg not downloaded yet
2026-04-01 08:17:22 [autobid.de] INFO: Page 5/141: found 20 items
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361290%27 HTTP/1.1" 200 None
2026-04-01 08:17:22 [autobid.de] INFO: Scrape type for 3361290: 1
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361291%27 HTTP/1.1" 200 None
2026-04-01 08:17:22 [autobid.de] INFO: Scrape type for 3361291: 1
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361292%27 HTTP/1.1" 200 None
2026-04-01 08:17:22 [autobid.de] INFO: Scrape type for 3361292: 1
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361293%27 HTTP/1.1" 200 None
2026-04-01 08:17:22 [autobid.de] INFO: Scrape type for 3361293: 1
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361294%27 HTTP/1.1" 200 None
2026-04-01 08:17:22 [autobid.de] INFO: Scrape type for 3361294: 1
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361295%27 HTTP/1.1" 200 None
2026-04-01 08:17:22 [autobid.de] INFO: Scrape type for 3361295: 1
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361296%27 HTTP/1.1" 200 None
2026-04-01 08:17:22 [autobid.de] INFO: Scrape type for 3361296: 1
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361297%27 HTTP/1.1" 200 None
2026-04-01 08:17:22 [autobid.de] INFO: Scrape type for 3361297: 1
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361298%27 HTTP/1.1" 200 None
2026-04-01 08:17:22 [autobid.de] INFO: Scrape type for 3361298: 1
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361299%27 HTTP/1.1" 200 None
2026-04-01 08:17:22 [autobid.de] INFO: Scrape type for 3361299: 1
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361300%27 HTTP/1.1" 200 None
2026-04-01 08:17:22 [autobid.de] INFO: Scrape type for 3361300: 1
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361301%27 HTTP/1.1" 200 None
2026-04-01 08:17:22 [autobid.de] INFO: Scrape type for 3361301: 1
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361302%27 HTTP/1.1" 200 None
2026-04-01 08:17:22 [autobid.de] INFO: Scrape type for 3361302: 1
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361303%27 HTTP/1.1" 200 None
2026-04-01 08:17:22 [autobid.de] INFO: Scrape type for 3361303: 1
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361304%27 HTTP/1.1" 200 None
2026-04-01 08:17:22 [autobid.de] INFO: Scrape type for 3361304: 1
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361305%27 HTTP/1.1" 200 None
2026-04-01 08:17:22 [autobid.de] INFO: Scrape type for 3361305: 1
2026-04-01 08:17:22 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 5c5b7dc0-0d5d-5a15-9dda-d2e5b8cc16ce with identifier: 3359679
2026-04-01 08:17:22 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 5c5b7dc0-0d5d-5a15-9dda-d2e5b8cc16ce with identifier: 3359679
2026-04-01 08:17:22 [scrapy.core.scraper] INFO: Dropped: Dropping expired listing: 5c5b7dc0-0d5d-5a15-9dda-d2e5b8cc16ce with expiration date: 2026-04-01T07:30:00Z
{'auction_title': 'Auction 81241',
 'body_type': 'Estate',
 'category': 'Estate',
 'color': 'black Pearl effect',
 'currency': 'EUR',
 'emission_standard': 'EURO 6',
 'engine': '2.0L',
 'engine_horse_power': 150,
 'expiration_date': '2026-04-01T07:30:00Z',
 'fuel_type': 'Diesel',
 'id': '5c5b7dc0-0d5d-5a15-9dda-d2e5b8cc16ce',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'link_to_web_offer': 'https://autobid.de/en/item/vw-passat-variant-2-0-tdi-bmt-dsg-comfortline-3359679',
 'listing': 'auction',
 'location': 'Oelde-Stromberg',
 'make': 'VW',
 'model': 'Passat Variant',
 'odometer': 242600,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 5,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33597/3359679/93534561_hd.jpg',
                'https://cdn.autobid.de/data/cars/33597/3359679/93534562_hd.jpg',
                'https://cdn.autobid.de/data/cars/33597/3359679/93534576_hd.jpg',
                'https://cdn.autobid.de/data/cars/33597/3359679/93534569_hd.jpg',
                'https://cdn.autobid.de/data/cars/33597/3359679/93534563_hd.jpg'],
 'photos': [],
 'price': 5150,
 'price_includes_vat': False,
 'registration_date': '2016-08-01',
 'seats_number': 5,
 'seller_name': 'Ulrich Senger GmbH',
 'title': 'VW Passat Variant 2,0 TDI BMT DSG "Comfortline"',
 'transmission': 'automatic',
 'vin': None,
 'year': 2016}
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359679%27 HTTP/1.1" 200 None
2026-04-01 08:17:22 [autobid.de] INFO: Saving data for 3359679: {'created_time': 1775031442.943276, 'last_price_update_time': 1775031442.943293, 'auction_closing_time': 1775028600.0}
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3359679') HTTP/1.1" 204 0
2026-04-01 08:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361306%27 HTTP/1.1" 200 None
2026-04-01 08:17:23 [autobid.de] INFO: Scrape type for 3361306: 1
2026-04-01 08:17:23 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 5c5b7dc0-0d5d-5a15-9dda-d2e5b8cc16ce due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33597/3359679/93534561_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361307%27 HTTP/1.1" 200 None
2026-04-01 08:17:23 [autobid.de] INFO: Scrape type for 3361307: 1
2026-04-01 08:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361308%27 HTTP/1.1" 200 None
2026-04-01 08:17:23 [autobid.de] INFO: Scrape type for 3361308: 1
2026-04-01 08:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361309%27 HTTP/1.1" 200 None
2026-04-01 08:17:23 [autobid.de] INFO: Scrape type for 3361309: 1
2026-04-01 08:17:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-t6-1-kombi-2-0-tdi-4motion-dsg-3361288/details> (referer: None)
2026-04-01 08:17:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 6d7cba0f-dc63-50f0-82c9-c6dbfd3fadd0 with identifier: 3361288
2026-04-01 08:17:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6d7cba0f-dc63-50f0-82c9-c6dbfd3fadd0 with scrape type 1
2026-04-01 08:17:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 6d7cba0f-dc63-50f0-82c9-c6dbfd3fadd0
2026-04-01 08:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33613%252F3361288%252F93419884_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:17:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33613/3361288/93419884_hd.jpg not downloaded yet
2026-04-01 08:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33613%252F3361288%252F93419889_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:17:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33613/3361288/93419889_hd.jpg not downloaded yet
2026-04-01 08:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33613%252F3361288%252F93419883_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:17:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33613/3361288/93419883_hd.jpg not downloaded yet
2026-04-01 08:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33613%252F3361288%252F93419898_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:17:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33613/3361288/93419898_hd.jpg not downloaded yet
2026-04-01 08:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33613%252F3361288%252F93419895_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:17:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33613/3361288/93419895_hd.jpg not downloaded yet
2026-04-01 08:17:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 4f0de064-e711-5575-9166-3d18fba5bf5d due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360079/93588688_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:17:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-id-buzz-pro-3361300/details> (referer: None)
2026-04-01 08:17:24 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 9718693b-102e-5038-b774-77a53d7a4f91 with identifier: 3361300
2026-04-01 08:17:24 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9718693b-102e-5038-b774-77a53d7a4f91 with scrape type 1
2026-04-01 08:17:24 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 9718693b-102e-5038-b774-77a53d7a4f91
2026-04-01 08:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361300%252F93421127_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:17:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361300/93421127_hd.jpg not downloaded yet
2026-04-01 08:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361300%252F93421124_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:17:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361300/93421124_hd.jpg not downloaded yet
2026-04-01 08:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361300%252F93421141_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:17:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361300/93421141_hd.jpg not downloaded yet
2026-04-01 08:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361300%252F93421121_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:17:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361300/93421121_hd.jpg not downloaded yet
2026-04-01 08:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361300%252F93421133_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:17:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361300/93421133_hd.jpg not downloaded yet
2026-04-01 08:17:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 4f0de064-e711-5575-9166-3d18fba5bf5d due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360079/93588687_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:17:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 4f0de064-e711-5575-9166-3d18fba5bf5d due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360079/93588686_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:17:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=6> (referer: None)
2026-04-01 08:17:27 [autobid.de] INFO: Page 6/141: found 20 items
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361310%27 HTTP/1.1" 200 None
2026-04-01 08:17:27 [autobid.de] INFO: Scrape type for 3361310: 1
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362455%27 HTTP/1.1" 200 None
2026-04-01 08:17:27 [autobid.de] INFO: Scrape type for 3362455: 1
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362995%27 HTTP/1.1" 200 None
2026-04-01 08:17:27 [autobid.de] INFO: Scrape type for 3362995: 1
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363011%27 HTTP/1.1" 200 None
2026-04-01 08:17:27 [autobid.de] INFO: Scrape type for 3363011: 1
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363701%27 HTTP/1.1" 200 None
2026-04-01 08:17:27 [autobid.de] INFO: Scrape type for 3363701: 1
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364667%27 HTTP/1.1" 200 None
2026-04-01 08:17:27 [autobid.de] INFO: Scrape type for 3364667: 1
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364668%27 HTTP/1.1" 200 None
2026-04-01 08:17:27 [autobid.de] INFO: Scrape type for 3364668: 1
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345117%27 HTTP/1.1" 200 None
2026-04-01 08:17:27 [autobid.de] INFO: Scrape type for 3345117: 1
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345123%27 HTTP/1.1" 200 None
2026-04-01 08:17:27 [autobid.de] INFO: Scrape type for 3345123: 1
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345125%27 HTTP/1.1" 200 None
2026-04-01 08:17:27 [autobid.de] INFO: Scrape type for 3345125: 1
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359075%27 HTTP/1.1" 200 None
2026-04-01 08:17:27 [autobid.de] INFO: Scrape type for 3359075: 1
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359076%27 HTTP/1.1" 200 None
2026-04-01 08:17:27 [autobid.de] INFO: Scrape type for 3359076: 1
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359077%27 HTTP/1.1" 200 None
2026-04-01 08:17:27 [autobid.de] INFO: Scrape type for 3359077: 1
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345121%27 HTTP/1.1" 200 None
2026-04-01 08:17:27 [autobid.de] INFO: Scrape type for 3345121: 1
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345126%27 HTTP/1.1" 200 None
2026-04-01 08:17:27 [autobid.de] INFO: Scrape type for 3345126: 1
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365053%27 HTTP/1.1" 200 None
2026-04-01 08:17:27 [autobid.de] INFO: Scrape type for 3365053: 1
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362878%27 HTTP/1.1" 200 None
2026-04-01 08:17:27 [autobid.de] INFO: Scrape type for 3362878: 1
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365130%27 HTTP/1.1" 200 None
2026-04-01 08:17:27 [autobid.de] INFO: Scrape type for 3365130: 1
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359994%27 HTTP/1.1" 200 None
2026-04-01 08:17:27 [autobid.de] INFO: Scrape type for 3359994: 1
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359995%27 HTTP/1.1" 200 None
2026-04-01 08:17:27 [autobid.de] INFO: Scrape type for 3359995: 1
2026-04-01 08:18:07 [scrapy.extensions.logstats] INFO: Crawled 31 pages (at 8 pages/min), scraped 1 items (at 0 items/min)
2026-04-01 08:18:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 197MiB
2026-04-01 08:19:07 [scrapy.extensions.logstats] INFO: Crawled 31 pages (at 0 pages/min), scraped 1 items (at 0 items/min)
2026-04-01 08:19:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 197MiB
2026-04-01 08:20:07 [scrapy.extensions.logstats] INFO: Crawled 31 pages (at 0 pages/min), scraped 1 items (at 0 items/min)
2026-04-01 08:20:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 197MiB
2026-04-01 08:20:16 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 4f0de064-e711-5575-9166-3d18fba5bf5d with identifier: 3360079
2026-04-01 08:20:16 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 4f0de064-e711-5575-9166-3d18fba5bf5d with identifier: 3360079
2026-04-01 08:20:16 [scrapy.core.scraper] INFO: Dropped: Dropping expired listing: 4f0de064-e711-5575-9166-3d18fba5bf5d with expiration date: 2026-04-01T07:30:00Z
{'auction_title': 'Auction 81241',
 'body_type': 'Estate',
 'category': 'Estate',
 'color': 'black Pearl effect',
 'currency': 'EUR',
 'emission_standard': 'EURO 6d-Temp',
 'engine': '1.5L',
 'engine_horse_power': 150,
 'expiration_date': '2026-04-01T07:30:00Z',
 'fuel_type': 'Petrol',
 'id': '4f0de064-e711-5575-9166-3d18fba5bf5d',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'link_to_web_offer': 'https://autobid.de/en/item/skoda-octavia-combi-1-5-tsi-ambition-3360079',
 'listing': 'auction',
 'location': 'Lübeck',
 'make': 'Skoda',
 'model': 'Octavia Combi',
 'odometer': 112500,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 5,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33601/3360079/93588685_hd.jpg',
                'https://cdn.autobid.de/data/cars/33601/3360079/93588686_hd.jpg',
                'https://cdn.autobid.de/data/cars/33601/3360079/93588687_hd.jpg',
                'https://cdn.autobid.de/data/cars/33601/3360079/93588688_hd.jpg',
                'https://cdn.autobid.de/data/cars/33601/3360079/93588689_hd.jpg'],
 'photos': [],
 'price': 10650,
 'price_includes_vat': False,
 'registration_date': '2019-09-01',
 'seats_number': 5,
 'seller_name': 'Senger Automobile GmbH',
 'title': 'Skoda Octavia Combi 1,5 TSI "Ambition"',
 'transmission': 'manual',
 'vin': None,
 'year': 2019}
2026-04-01 08:20:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360079%27 HTTP/1.1" 200 None
2026-04-01 08:20:16 [autobid.de] INFO: Saving data for 3360079: {'created_time': 1775031616.622759, 'last_price_update_time': 1775031616.62277, 'auction_closing_time': 1775028600.0}
2026-04-01 08:20:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:16 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3360079') HTTP/1.1" 204 0
2026-04-01 08:20:16 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 4f0de064-e711-5575-9166-3d18fba5bf5d due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360079/93588685_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:20:18 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 3566bbdb-3e02-578f-a4fb-09d37897e274 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360080/93590987_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:20:18 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/seat-leon-st-1-8-tsi-4drive-dsg-x-perience-3359995/details> (referer: None)
2026-04-01 08:20:18 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 153da03e-21e1-5199-9b3c-4931d86e694d with identifier: 3359995
2026-04-01 08:20:18 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 153da03e-21e1-5199-9b3c-4931d86e694d with scrape type 1
2026-04-01 08:20:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 153da03e-21e1-5199-9b3c-4931d86e694d
2026-04-01 08:20:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33600%252F3359995%252F93500035_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33600/3359995/93500035_hd.jpg not downloaded yet
2026-04-01 08:20:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33600%252F3359995%252F93500036_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33600/3359995/93500036_hd.jpg not downloaded yet
2026-04-01 08:20:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33600%252F3359995%252F93500037_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33600/3359995/93500037_hd.jpg not downloaded yet
2026-04-01 08:20:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33600%252F3359995%252F93500038_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33600/3359995/93500038_hd.jpg not downloaded yet
2026-04-01 08:20:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33600%252F3359995%252F93500039_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33600/3359995/93500039_hd.jpg not downloaded yet
2026-04-01 08:20:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=7> (referer: None)
2026-04-01 08:20:19 [autobid.de] INFO: Page 7/142: found 20 items
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361694%27 HTTP/1.1" 200 None
2026-04-01 08:20:19 [autobid.de] INFO: Scrape type for 3361694: 1
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362456%27 HTTP/1.1" 200 None
2026-04-01 08:20:19 [autobid.de] INFO: Scrape type for 3362456: 1
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361699%27 HTTP/1.1" 200 None
2026-04-01 08:20:19 [autobid.de] INFO: Scrape type for 3361699: 1
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360558%27 HTTP/1.1" 200 None
2026-04-01 08:20:19 [autobid.de] INFO: Scrape type for 3360558: 1
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360559%27 HTTP/1.1" 200 None
2026-04-01 08:20:19 [autobid.de] INFO: Scrape type for 3360559: 1
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361696%27 HTTP/1.1" 200 None
2026-04-01 08:20:19 [autobid.de] INFO: Scrape type for 3361696: 1
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360563%27 HTTP/1.1" 200 None
2026-04-01 08:20:19 [autobid.de] INFO: Scrape type for 3360563: 1
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364673%27 HTTP/1.1" 200 None
2026-04-01 08:20:19 [autobid.de] INFO: Scrape type for 3364673: 1
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364674%27 HTTP/1.1" 200 None
2026-04-01 08:20:19 [autobid.de] INFO: Scrape type for 3364674: 1
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364675%27 HTTP/1.1" 200 None
2026-04-01 08:20:19 [autobid.de] INFO: Scrape type for 3364675: 1
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364676%27 HTTP/1.1" 200 None
2026-04-01 08:20:19 [autobid.de] INFO: Scrape type for 3364676: 1
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364677%27 HTTP/1.1" 200 None
2026-04-01 08:20:19 [autobid.de] INFO: Scrape type for 3364677: 1
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364678%27 HTTP/1.1" 200 None
2026-04-01 08:20:19 [autobid.de] INFO: Scrape type for 3364678: 1
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364679%27 HTTP/1.1" 200 None
2026-04-01 08:20:19 [autobid.de] INFO: Scrape type for 3364679: 1
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364680%27 HTTP/1.1" 200 None
2026-04-01 08:20:19 [autobid.de] INFO: Scrape type for 3364680: 1
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364681%27 HTTP/1.1" 200 None
2026-04-01 08:20:19 [autobid.de] INFO: Scrape type for 3364681: 1
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364682%27 HTTP/1.1" 200 None
2026-04-01 08:20:19 [autobid.de] INFO: Scrape type for 3364682: 1
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364683%27 HTTP/1.1" 200 None
2026-04-01 08:20:19 [autobid.de] INFO: Scrape type for 3364683: 1
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364684%27 HTTP/1.1" 200 None
2026-04-01 08:20:19 [autobid.de] INFO: Scrape type for 3364684: 1
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364685%27 HTTP/1.1" 200 None
2026-04-01 08:20:19 [autobid.de] INFO: Scrape type for 3364685: 1
2026-04-01 08:20:19 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 3566bbdb-3e02-578f-a4fb-09d37897e274 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360080/93590986_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:20:19 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item da10067d-0f0f-5425-845f-788747dea6b3 due to An error occurred while connecting: 104: Connection reset by peer.
2026-04-01 08:20:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/hyundai-tucson-1-6-t-gdi-2wd-passion-3359994/details> (referer: None)
2026-04-01 08:20:19 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 949471e8-ed42-5978-8cc1-d997eca79b31 with identifier: 3359994
2026-04-01 08:20:19 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 949471e8-ed42-5978-8cc1-d997eca79b31 with scrape type 1
2026-04-01 08:20:19 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 949471e8-ed42-5978-8cc1-d997eca79b31
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33600%252F3359994%252F93494548_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33600/3359994/93494548_hd.jpg not downloaded yet
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33600%252F3359994%252F93494549_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33600/3359994/93494549_hd.jpg not downloaded yet
2026-04-01 08:20:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33600%252F3359994%252F93494550_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33600/3359994/93494550_hd.jpg not downloaded yet
2026-04-01 08:20:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33600%252F3359994%252F93494551_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33600/3359994/93494551_hd.jpg not downloaded yet
2026-04-01 08:20:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33600%252F3359994%252F93494552_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33600/3359994/93494552_hd.jpg not downloaded yet
2026-04-01 08:20:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 3566bbdb-3e02-578f-a4fb-09d37897e274 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360080/93590985_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:20:21 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ford-focus-turnier-1-5-ecoblue-tdci-automatik-st-line-3364683/details> (referer: None)
2026-04-01 08:20:21 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: a851f583-7912-59fe-965f-2d6a0a222d86 with identifier: 3364683
2026-04-01 08:20:21 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a851f583-7912-59fe-965f-2d6a0a222d86 with scrape type 1
2026-04-01 08:20:21 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item a851f583-7912-59fe-965f-2d6a0a222d86
2026-04-01 08:20:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364683%252F93543707_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364683/93543707_hd.jpg not downloaded yet
2026-04-01 08:20:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364683%252F93543696_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364683/93543696_hd.jpg not downloaded yet
2026-04-01 08:20:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364683%252F93543712_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364683/93543712_hd.jpg not downloaded yet
2026-04-01 08:20:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364683%252F93543706_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364683/93543706_hd.jpg not downloaded yet
2026-04-01 08:20:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364683%252F93543688_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364683/93543688_hd.jpg not downloaded yet
2026-04-01 08:20:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ford-mondeo-turnier-1-5-ecoboost-titanium-3364684/details> (referer: None)
2026-04-01 08:20:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 7c217fdb-b164-5b5d-8d21-ed7086eda7f5 with identifier: 3364684
2026-04-01 08:20:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 7c217fdb-b164-5b5d-8d21-ed7086eda7f5 with scrape type 1
2026-04-01 08:20:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 7c217fdb-b164-5b5d-8d21-ed7086eda7f5
2026-04-01 08:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364684%252F93543733_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364684/93543733_hd.jpg not downloaded yet
2026-04-01 08:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364684%252F93543748_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364684/93543748_hd.jpg not downloaded yet
2026-04-01 08:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364684%252F93543743_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364684/93543743_hd.jpg not downloaded yet
2026-04-01 08:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364684%252F93543742_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364684/93543742_hd.jpg not downloaded yet
2026-04-01 08:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364684%252F93543744_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364684/93543744_hd.jpg not downloaded yet
2026-04-01 08:20:22 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 3566bbdb-3e02-578f-a4fb-09d37897e274 with identifier: 3360080
2026-04-01 08:20:22 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 08:20:23 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=Diesel+Mildhybrid HTTP/1.1" 200 None
2026-04-01 08:20:23 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 08:20:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 08:20:23 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 3566bbdb-3e02-578f-a4fb-09d37897e274 with identifier: 3360080
2026-04-01 08:20:23 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: 3566bbdb-3e02-578f-a4fb-09d37897e274 with identifier: 3360080 to the API
2026-04-01 08:20:23 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 08:20:23 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 08:20:23 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360080/93590984_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 81241',
 'body_type': 'Estate',
 'category': 'Estate',
 'color': 'grey Metallic',
 'currency': 'EUR',
 'emission_standard': 'EURO 6d-Temp',
 'engine': '2.0L',
 'engine_horse_power': 163,
 'fuel_type': 'Diesel Mild Hybrid',
 'id': '3566bbdb-3e02-578f-a4fb-09d37897e274',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_included': True,
 'link_to_web_offer': 'https://autobid.de/en/item/audi-a4-avant-35-tdi-s-tronic-3360080',
 'listing': 'auction',
 'location': 'Lübeck',
 'make': 'Audi',
 'model': 'A4 Avant',
 'odometer': 111900,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 5,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33601/3360080/93590984_hd.jpg',
                'https://cdn.autobid.de/data/cars/33601/3360080/93590985_hd.jpg',
                'https://cdn.autobid.de/data/cars/33601/3360080/93590986_hd.jpg',
                'https://cdn.autobid.de/data/cars/33601/3360080/93590987_hd.jpg',
                'https://cdn.autobid.de/data/cars/33601/3360080/93590988_hd.jpg'],
 'photos': [],
 'price': 15050,
 'price_includes_vat': True,
 'registration_date': '2022-05-01',
 'seats_number': 5,
 'seller_name': 'Senger Automobile GmbH',
 'title': 'Audi A4 Avant 35 TDI S-tronic',
 'transmission': 'automatic',
 'vin': None,
 'year': 2022}
2026-04-01 08:20:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360080%27 HTTP/1.1" 200 None
2026-04-01 08:20:23 [autobid.de] INFO: Saving data for 3360080: {'created_time': 1775031623.47542, 'last_price_update_time': 1775031623.475432}
2026-04-01 08:20:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3360080') HTTP/1.1" 204 0
2026-04-01 08:20:23 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 3566bbdb-3e02-578f-a4fb-09d37897e274 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360080/93590984_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:20:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 976070b1-c53a-5f93-be5c-6206108a7858 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33630/3362931/93586473_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:20:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 2f8544d1-65bf-566a-8914-8d7fe64bd511 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362684/93483104_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:20:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=8> (referer: None)
2026-04-01 08:20:26 [autobid.de] INFO: Page 8/142: found 20 items
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364707%27 HTTP/1.1" 200 None
2026-04-01 08:20:26 [autobid.de] INFO: Scrape type for 3364707: 1
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364687%27 HTTP/1.1" 200 None
2026-04-01 08:20:26 [autobid.de] INFO: Scrape type for 3364687: 1
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365073%27 HTTP/1.1" 200 None
2026-04-01 08:20:26 [autobid.de] INFO: Scrape type for 3365073: 1
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365074%27 HTTP/1.1" 200 None
2026-04-01 08:20:26 [autobid.de] INFO: Scrape type for 3365074: 1
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365075%27 HTTP/1.1" 200 None
2026-04-01 08:20:26 [autobid.de] INFO: Scrape type for 3365075: 1
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365076%27 HTTP/1.1" 200 None
2026-04-01 08:20:26 [autobid.de] INFO: Scrape type for 3365076: 1
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365077%27 HTTP/1.1" 200 None
2026-04-01 08:20:26 [autobid.de] INFO: Scrape type for 3365077: 1
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365078%27 HTTP/1.1" 200 None
2026-04-01 08:20:26 [autobid.de] INFO: Scrape type for 3365078: 1
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365079%27 HTTP/1.1" 200 None
2026-04-01 08:20:26 [autobid.de] INFO: Scrape type for 3365079: 1
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364686%27 HTTP/1.1" 200 None
2026-04-01 08:20:26 [autobid.de] INFO: Scrape type for 3364686: 1
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365080%27 HTTP/1.1" 200 None
2026-04-01 08:20:26 [autobid.de] INFO: Scrape type for 3365080: 1
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365081%27 HTTP/1.1" 200 None
2026-04-01 08:20:26 [autobid.de] INFO: Scrape type for 3365081: 1
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365084%27 HTTP/1.1" 200 None
2026-04-01 08:20:26 [autobid.de] INFO: Scrape type for 3365084: 1
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365085%27 HTTP/1.1" 200 None
2026-04-01 08:20:26 [autobid.de] INFO: Scrape type for 3365085: 1
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365565%27 HTTP/1.1" 200 None
2026-04-01 08:20:26 [autobid.de] INFO: Scrape type for 3365565: 1
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365566%27 HTTP/1.1" 200 None
2026-04-01 08:20:26 [autobid.de] INFO: Scrape type for 3365566: 1
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365567%27 HTTP/1.1" 200 None
2026-04-01 08:20:26 [autobid.de] INFO: Scrape type for 3365567: 1
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365568%27 HTTP/1.1" 200 None
2026-04-01 08:20:26 [autobid.de] INFO: Scrape type for 3365568: 1
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365569%27 HTTP/1.1" 200 None
2026-04-01 08:20:26 [autobid.de] INFO: Scrape type for 3365569: 1
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360560%27 HTTP/1.1" 200 None
2026-04-01 08:20:26 [autobid.de] INFO: Scrape type for 3360560: 1
2026-04-01 08:20:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 07dab192-7e58-593b-9e59-cc2c85c84481 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33576/3357570/93575111_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:20:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=9> (referer: None)
2026-04-01 08:20:29 [autobid.de] INFO: Page 9/142: found 20 items
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360564%27 HTTP/1.1" 200 None
2026-04-01 08:20:29 [autobid.de] INFO: Scrape type for 3360564: 1
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360549%27 HTTP/1.1" 200 None
2026-04-01 08:20:29 [autobid.de] INFO: Scrape type for 3360549: 1
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360550%27 HTTP/1.1" 200 None
2026-04-01 08:20:29 [autobid.de] INFO: Scrape type for 3360550: 1
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360553%27 HTTP/1.1" 200 None
2026-04-01 08:20:29 [autobid.de] INFO: Scrape type for 3360553: 1
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360551%27 HTTP/1.1" 200 None
2026-04-01 08:20:29 [autobid.de] INFO: Scrape type for 3360551: 1
2026-04-01 08:20:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ford-focus-turnier-1-5-ecoblue-tdci-automatik-titanium-x-3360560/details> (referer: None)
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360552%27 HTTP/1.1" 200 None
2026-04-01 08:20:29 [autobid.de] INFO: Scrape type for 3360552: 1
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360556%27 HTTP/1.1" 200 None
2026-04-01 08:20:29 [autobid.de] INFO: Scrape type for 3360556: 1
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360554%27 HTTP/1.1" 200 None
2026-04-01 08:20:29 [autobid.de] INFO: Scrape type for 3360554: 1
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360555%27 HTTP/1.1" 200 None
2026-04-01 08:20:29 [autobid.de] INFO: Scrape type for 3360555: 1
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360557%27 HTTP/1.1" 200 None
2026-04-01 08:20:29 [autobid.de] INFO: Scrape type for 3360557: 1
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360561%27 HTTP/1.1" 200 None
2026-04-01 08:20:29 [autobid.de] INFO: Scrape type for 3360561: 1
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361697%27 HTTP/1.1" 200 None
2026-04-01 08:20:29 [autobid.de] INFO: Scrape type for 3361697: 1
2026-04-01 08:20:29 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 2eda1022-1b8f-5af0-930f-cce17041fb7b with identifier: 3360560
2026-04-01 08:20:29 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 2eda1022-1b8f-5af0-930f-cce17041fb7b with scrape type 1
2026-04-01 08:20:29 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 2eda1022-1b8f-5af0-930f-cce17041fb7b
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33606%252F3360560%252F93609767_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33606/3360560/93609767_hd.jpg not downloaded yet
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361698%27 HTTP/1.1" 200 None
2026-04-01 08:20:29 [autobid.de] INFO: Scrape type for 3361698: 1
2026-04-01 08:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33606%252F3360560%252F93609770_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33606/3360560/93609770_hd.jpg not downloaded yet
2026-04-01 08:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366524%27 HTTP/1.1" 200 None
2026-04-01 08:20:30 [autobid.de] INFO: Scrape type for 3366524: 1
2026-04-01 08:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33606%252F3360560%252F93609785_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33606/3360560/93609785_hd.jpg not downloaded yet
2026-04-01 08:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366525%27 HTTP/1.1" 200 None
2026-04-01 08:20:30 [autobid.de] INFO: Scrape type for 3366525: 1
2026-04-01 08:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33606%252F3360560%252F93609788_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33606/3360560/93609788_hd.jpg not downloaded yet
2026-04-01 08:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366531%27 HTTP/1.1" 200 None
2026-04-01 08:20:30 [autobid.de] INFO: Scrape type for 3366531: 1
2026-04-01 08:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33606%252F3360560%252F93609782_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33606/3360560/93609782_hd.jpg not downloaded yet
2026-04-01 08:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360135%27 HTTP/1.1" 200 None
2026-04-01 08:20:30 [autobid.de] INFO: Scrape type for 3360135: 1
2026-04-01 08:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361695%27 HTTP/1.1" 200 None
2026-04-01 08:20:30 [autobid.de] INFO: Scrape type for 3361695: 1
2026-04-01 08:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361692%27 HTTP/1.1" 200 None
2026-04-01 08:20:30 [autobid.de] INFO: Scrape type for 3361692: 1
2026-04-01 08:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360141%27 HTTP/1.1" 200 None
2026-04-01 08:20:30 [autobid.de] INFO: Scrape type for 3360141: 1
2026-04-01 08:20:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ford-transit-doka-pritsche-2-0-tdci-310-l2-3360553/details> (referer: None)
2026-04-01 08:20:31 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: c7715c5a-8bd3-54eb-aaf5-b7dcf94eee20 with identifier: 3360553
2026-04-01 08:20:31 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item c7715c5a-8bd3-54eb-aaf5-b7dcf94eee20 with scrape type 1
2026-04-01 08:20:31 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item c7715c5a-8bd3-54eb-aaf5-b7dcf94eee20
2026-04-01 08:20:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33606%252F3360553%252F93618600_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33606/3360553/93618600_hd.jpg not downloaded yet
2026-04-01 08:20:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33606%252F3360553%252F93618601_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33606/3360553/93618601_hd.jpg not downloaded yet
2026-04-01 08:20:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33606%252F3360553%252F93618602_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33606/3360553/93618602_hd.jpg not downloaded yet
2026-04-01 08:20:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33606%252F3360553%252F93618603_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33606/3360553/93618603_hd.jpg not downloaded yet
2026-04-01 08:20:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33606%252F3360553%252F93618604_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:20:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33606/3360553/93618604_hd.jpg not downloaded yet
2026-04-01 08:20:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=10> (referer: None)
2026-04-01 08:20:33 [autobid.de] INFO: Page 10/142: found 20 items
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361700%27 HTTP/1.1" 200 None
2026-04-01 08:20:33 [autobid.de] INFO: Scrape type for 3361700: 1
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360140%27 HTTP/1.1" 200 None
2026-04-01 08:20:33 [autobid.de] INFO: Scrape type for 3360140: 1
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366554%27 HTTP/1.1" 200 None
2026-04-01 08:20:33 [autobid.de] INFO: Scrape type for 3366554: 1
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366555%27 HTTP/1.1" 200 None
2026-04-01 08:20:33 [autobid.de] INFO: Scrape type for 3366555: 1
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366556%27 HTTP/1.1" 200 None
2026-04-01 08:20:33 [autobid.de] INFO: Scrape type for 3366556: 1
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366820%27 HTTP/1.1" 200 None
2026-04-01 08:20:33 [autobid.de] INFO: Scrape type for 3366820: 1
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366821%27 HTTP/1.1" 200 None
2026-04-01 08:20:33 [autobid.de] INFO: Scrape type for 3366821: 1
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366822%27 HTTP/1.1" 200 None
2026-04-01 08:20:33 [autobid.de] INFO: Scrape type for 3366822: 1
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356331%27 HTTP/1.1" 200 None
2026-04-01 08:20:33 [autobid.de] INFO: Scrape type for 3356331: 1
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364619%27 HTTP/1.1" 200 None
2026-04-01 08:20:33 [autobid.de] INFO: Scrape type for 3364619: 1
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347977%27 HTTP/1.1" 200 None
2026-04-01 08:20:33 [autobid.de] INFO: Scrape type for 3347977: 1
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364618%27 HTTP/1.1" 200 None
2026-04-01 08:20:33 [autobid.de] INFO: Scrape type for 3364618: 1
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352453%27 HTTP/1.1" 200 None
2026-04-01 08:20:33 [autobid.de] INFO: Scrape type for 3352453: 1
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364606%27 HTTP/1.1" 200 None
2026-04-01 08:20:33 [autobid.de] INFO: Scrape type for 3364606: 1
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354809%27 HTTP/1.1" 200 None
2026-04-01 08:20:33 [autobid.de] INFO: Scrape type for 3354809: 1
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364629%27 HTTP/1.1" 200 None
2026-04-01 08:20:33 [autobid.de] INFO: Scrape type for 3364629: 1
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355045%27 HTTP/1.1" 200 None
2026-04-01 08:20:33 [autobid.de] INFO: Scrape type for 3355045: 1
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364605%27 HTTP/1.1" 200 None
2026-04-01 08:20:33 [autobid.de] INFO: Scrape type for 3364605: 1
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359084%27 HTTP/1.1" 200 None
2026-04-01 08:20:33 [autobid.de] INFO: Scrape type for 3359084: 1
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364622%27 HTTP/1.1" 200 None
2026-04-01 08:20:33 [autobid.de] INFO: Scrape type for 3364622: 1
2026-04-01 08:21:07 [scrapy.extensions.logstats] INFO: Crawled 41 pages (at 10 pages/min), scraped 2 items (at 1 items/min)
2026-04-01 08:21:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 224MiB
2026-04-01 08:22:07 [scrapy.extensions.logstats] INFO: Crawled 41 pages (at 0 pages/min), scraped 2 items (at 0 items/min)
2026-04-01 08:22:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 224MiB
2026-04-01 08:23:07 [scrapy.extensions.logstats] INFO: Crawled 41 pages (at 0 pages/min), scraped 2 items (at 0 items/min)
2026-04-01 08:23:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 224MiB
2026-04-01 08:23:16 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item b368875c-8f77-5680-9087-a40250631e69 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33664/3366371/93614075_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:23:18 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: da10067d-0f0f-5425-845f-788747dea6b3 with identifier: 3366376
2026-04-01 08:23:18 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: da10067d-0f0f-5425-845f-788747dea6b3 with identifier: 3366376
2026-04-01 08:23:18 [scrapy.core.scraper] INFO: Dropped: Dropping expired listing: da10067d-0f0f-5425-845f-788747dea6b3 with expiration date: 2026-04-01T08:00:00Z
{'auction_title': 'Auction 81309',
 'body_type': 'Van / minibus',
 'category': 'Van / minibus',
 'color': 'silver Metallic',
 'currency': 'EUR',
 'emission_standard': 'EURO 6',
 'engine': '1.4L',
 'engine_horse_power': 150,
 'expiration_date': '2026-04-01T08:00:00Z',
 'fuel_type': 'Petrol',
 'id': 'da10067d-0f0f-5425-845f-788747dea6b3',
 'is_damaged': True,
 'is_operable': True,
 'is_vat_deductible': False,
 'link_to_web_offer': 'https://autobid.de/en/item/vw-touran-comfortline-1-4-tsi-acc-ahk-front-16-3366376',
 'listing': 'auction',
 'location': 'Fürstenfeldbruck',
 'make': 'VW',
 'model': 'Touran Comfortline',
 'odometer': 78900,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 2,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33664/3366376/93616550_hd.jpg',
                'https://cdn.autobid.de/data/cars/33664/3366376/93616544_hd.jpg'],
 'photos': [],
 'price': 14000,
 'price_includes_vat': False,
 'registration_date': '2017-11-01',
 'seller_name': 'Subsidiary/authorised dealer of the brand(s): Audi',
 'title': "VW Touran Comfortline 1.4 TSI *ACC*AHK*FRONT*16''*",
 'transmission': 'automatic',
 'vin': None,
 'year': 2017}
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366376%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [autobid.de] INFO: Saving data for 3366376: {'created_time': 1775031798.108234, 'last_price_update_time': 1775031798.108245, 'auction_closing_time': 1775030400.0}
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3366376') HTTP/1.1" 204 0
2026-04-01 08:23:18 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item da10067d-0f0f-5425-845f-788747dea6b3 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33664/3366376/93616544_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:23:18 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=11> (referer: None)
2026-04-01 08:23:18 [autobid.de] INFO: Page 11/142: found 20 items
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363729%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [autobid.de] INFO: Scrape type for 3363729: 1
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364621%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [autobid.de] INFO: Scrape type for 3364621: 1
2026-04-01 08:23:18 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/jaguar-xf-3-0-v6-diesel-automatik-luxury-3364622/details> (referer: None)
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363730%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [autobid.de] INFO: Scrape type for 3363730: 1
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364637%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [autobid.de] INFO: Scrape type for 3364637: 1
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363731%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [autobid.de] INFO: Scrape type for 3363731: 1
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364642%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [autobid.de] INFO: Scrape type for 3364642: 1
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365707%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [autobid.de] INFO: Scrape type for 3365707: 1
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364612%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [autobid.de] INFO: Scrape type for 3364612: 1
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365708%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [autobid.de] INFO: Scrape type for 3365708: 1
2026-04-01 08:23:18 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: f392b3c0-1e10-595f-89e9-5addfdc26094 with identifier: 3364622
2026-04-01 08:23:18 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f392b3c0-1e10-595f-89e9-5addfdc26094 with scrape type 1
2026-04-01 08:23:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item f392b3c0-1e10-595f-89e9-5addfdc26094
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364622%252F93541137_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364622/93541137_hd.jpg not downloaded yet
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364688%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [autobid.de] INFO: Scrape type for 3364688: 1
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364622%252F93541167_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364622/93541167_hd.jpg not downloaded yet
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365709%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [autobid.de] INFO: Scrape type for 3365709: 1
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364622%252F93541139_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364622/93541139_hd.jpg not downloaded yet
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364739%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [autobid.de] INFO: Scrape type for 3364739: 1
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364622%252F93541138_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364622/93541138_hd.jpg not downloaded yet
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365710%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [autobid.de] INFO: Scrape type for 3365710: 1
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364622%252F93541166_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364622/93541166_hd.jpg not downloaded yet
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364738%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [autobid.de] INFO: Scrape type for 3364738: 1
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365711%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [autobid.de] INFO: Scrape type for 3365711: 1
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364711%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [autobid.de] INFO: Scrape type for 3364711: 1
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361470%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [autobid.de] INFO: Scrape type for 3361470: 1
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364745%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [autobid.de] INFO: Scrape type for 3364745: 1
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361744%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [autobid.de] INFO: Scrape type for 3361744: 1
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364743%27 HTTP/1.1" 200 None
2026-04-01 08:23:18 [autobid.de] INFO: Scrape type for 3364743: 1
2026-04-01 08:23:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-golf-2-0-tdi-dsg-comfortline-3359084/details> (referer: None)
2026-04-01 08:23:19 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 97ebdcf7-f035-5f0e-86d9-d477b0a86dca with identifier: 3359084
2026-04-01 08:23:19 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 97ebdcf7-f035-5f0e-86d9-d477b0a86dca with scrape type 1
2026-04-01 08:23:19 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 97ebdcf7-f035-5f0e-86d9-d477b0a86dca
2026-04-01 08:23:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359084%252F93452763_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:23:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359084/93452763_hd.jpg not downloaded yet
2026-04-01 08:23:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359084%252F93452764_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:23:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359084/93452764_hd.jpg not downloaded yet
2026-04-01 08:23:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359084%252F93452765_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:23:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359084/93452765_hd.jpg not downloaded yet
2026-04-01 08:23:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359084%252F93452766_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:23:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359084/93452766_hd.jpg not downloaded yet
2026-04-01 08:23:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359084%252F93452767_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:23:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359084/93452767_hd.jpg not downloaded yet
2026-04-01 08:23:20 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/bmw-318d-touring-advantage-3364621/details> (referer: None)
2026-04-01 08:23:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 6d7cba0f-dc63-50f0-82c9-c6dbfd3fadd0 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33613/3361288/93419895_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:23:20 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: a783c9f0-5f13-5a04-bc4e-01698d0b1311 with identifier: 3364621
2026-04-01 08:23:20 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a783c9f0-5f13-5a04-bc4e-01698d0b1311 with scrape type 1
2026-04-01 08:23:20 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item a783c9f0-5f13-5a04-bc4e-01698d0b1311
2026-04-01 08:23:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364621%252F93541080_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:23:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364621/93541080_hd.jpg not downloaded yet
2026-04-01 08:23:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364621%252F93541093_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:23:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364621/93541093_hd.jpg not downloaded yet
2026-04-01 08:23:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364621%252F93541104_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:23:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364621/93541104_hd.jpg not downloaded yet
2026-04-01 08:23:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364621%252F93541094_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:23:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364621/93541094_hd.jpg not downloaded yet
2026-04-01 08:23:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364621%252F93541079_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:23:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364621/93541079_hd.jpg not downloaded yet
2026-04-01 08:23:22 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 6d7cba0f-dc63-50f0-82c9-c6dbfd3fadd0 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33613/3361288/93419898_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:23:23 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 9718693b-102e-5038-b774-77a53d7a4f91 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33614/3361300/93421133_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:23:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=12> (referer: None)
2026-04-01 08:23:24 [autobid.de] INFO: Page 12/142: found 20 items
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362724%27 HTTP/1.1" 200 None
2026-04-01 08:23:24 [autobid.de] INFO: Scrape type for 3362724: 1
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364744%27 HTTP/1.1" 200 None
2026-04-01 08:23:24 [autobid.de] INFO: Scrape type for 3364744: 1
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359072%27 HTTP/1.1" 200 None
2026-04-01 08:23:24 [autobid.de] INFO: Scrape type for 3359072: 1
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364741%27 HTTP/1.1" 200 None
2026-04-01 08:23:24 [autobid.de] INFO: Scrape type for 3364741: 1
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359073%27 HTTP/1.1" 200 None
2026-04-01 08:23:24 [autobid.de] INFO: Scrape type for 3359073: 1
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364742%27 HTTP/1.1" 200 None
2026-04-01 08:23:24 [autobid.de] INFO: Scrape type for 3364742: 1
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359074%27 HTTP/1.1" 200 None
2026-04-01 08:23:24 [autobid.de] INFO: Scrape type for 3359074: 1
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364740%27 HTTP/1.1" 200 None
2026-04-01 08:23:24 [autobid.de] INFO: Scrape type for 3364740: 1
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359889%27 HTTP/1.1" 200 None
2026-04-01 08:23:24 [autobid.de] INFO: Scrape type for 3359889: 1
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364761%27 HTTP/1.1" 200 None
2026-04-01 08:23:24 [autobid.de] INFO: Scrape type for 3364761: 1
2026-04-01 08:23:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 9718693b-102e-5038-b774-77a53d7a4f91 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33614/3361300/93421121_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360051%27 HTTP/1.1" 200 None
2026-04-01 08:23:24 [autobid.de] INFO: Scrape type for 3360051: 1
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364710%27 HTTP/1.1" 200 None
2026-04-01 08:23:24 [autobid.de] INFO: Scrape type for 3364710: 1
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360519%27 HTTP/1.1" 200 None
2026-04-01 08:23:24 [autobid.de] INFO: Scrape type for 3360519: 1
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364626%27 HTTP/1.1" 200 None
2026-04-01 08:23:24 [autobid.de] INFO: Scrape type for 3364626: 1
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363763%27 HTTP/1.1" 200 None
2026-04-01 08:23:24 [autobid.de] INFO: Scrape type for 3363763: 1
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364640%27 HTTP/1.1" 200 None
2026-04-01 08:23:24 [autobid.de] INFO: Scrape type for 3364640: 1
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361813%27 HTTP/1.1" 200 None
2026-04-01 08:23:24 [autobid.de] INFO: Scrape type for 3361813: 1
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364759%27 HTTP/1.1" 200 None
2026-04-01 08:23:24 [autobid.de] INFO: Scrape type for 3364759: 1
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361743%27 HTTP/1.1" 200 None
2026-04-01 08:23:24 [autobid.de] INFO: Scrape type for 3361743: 1
2026-04-01 08:23:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364762%27 HTTP/1.1" 200 None
2026-04-01 08:23:25 [autobid.de] INFO: Scrape type for 3364762: 1
2026-04-01 08:23:25 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/bmw-530d-touring-xdrive-steptronic-m-sportpaket-3364741/details> (referer: None)
2026-04-01 08:23:25 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 479734c2-df31-5e26-a862-9b7d652bd054 with identifier: 3364741
2026-04-01 08:23:25 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 479734c2-df31-5e26-a862-9b7d652bd054 with scrape type 1
2026-04-01 08:23:25 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 479734c2-df31-5e26-a862-9b7d652bd054
2026-04-01 08:23:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33648%252F3364741%252F93546417_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:23:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33648/3364741/93546417_hd.jpg not downloaded yet
2026-04-01 08:23:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33648%252F3364741%252F93546411_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:23:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33648/3364741/93546411_hd.jpg not downloaded yet
2026-04-01 08:23:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33648%252F3364741%252F93546419_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:23:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33648/3364741/93546419_hd.jpg not downloaded yet
2026-04-01 08:23:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33648%252F3364741%252F93546414_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:23:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33648/3364741/93546414_hd.jpg not downloaded yet
2026-04-01 08:23:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33648%252F3364741%252F93546410_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:23:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33648/3364741/93546410_hd.jpg not downloaded yet
2026-04-01 08:23:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 9718693b-102e-5038-b774-77a53d7a4f91 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33614/3361300/93421141_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:23:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=13> (referer: None)
2026-04-01 08:23:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 153da03e-21e1-5199-9b3c-4931d86e694d due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33600/3359995/93500039_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:23:27 [autobid.de] INFO: Page 13/142: found 20 items
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354810%27 HTTP/1.1" 200 None
2026-04-01 08:23:27 [autobid.de] INFO: Scrape type for 3354810: 1
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364763%27 HTTP/1.1" 200 None
2026-04-01 08:23:27 [autobid.de] INFO: Scrape type for 3364763: 1
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354808%27 HTTP/1.1" 200 None
2026-04-01 08:23:27 [autobid.de] INFO: Scrape type for 3354808: 1
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365058%27 HTTP/1.1" 200 None
2026-04-01 08:23:27 [autobid.de] INFO: Scrape type for 3365058: 1
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352452%27 HTTP/1.1" 200 None
2026-04-01 08:23:27 [autobid.de] INFO: Scrape type for 3352452: 1
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365057%27 HTTP/1.1" 200 None
2026-04-01 08:23:27 [autobid.de] INFO: Scrape type for 3365057: 1
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273342539%27 HTTP/1.1" 200 None
2026-04-01 08:23:27 [autobid.de] INFO: Scrape type for 3342539: 1
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365059%27 HTTP/1.1" 200 None
2026-04-01 08:23:27 [autobid.de] INFO: Scrape type for 3365059: 1
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273325880%27 HTTP/1.1" 200 None
2026-04-01 08:23:27 [autobid.de] INFO: Scrape type for 3325880: 1
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365056%27 HTTP/1.1" 200 None
2026-04-01 08:23:27 [autobid.de] INFO: Scrape type for 3365056: 1
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366909%27 HTTP/1.1" 200 None
2026-04-01 08:23:27 [autobid.de] INFO: Scrape type for 3366909: 1
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365024%27 HTTP/1.1" 200 None
2026-04-01 08:23:27 [autobid.de] INFO: Scrape type for 3365024: 1
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366954%27 HTTP/1.1" 200 None
2026-04-01 08:23:27 [autobid.de] INFO: Scrape type for 3366954: 1
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365060%27 HTTP/1.1" 200 None
2026-04-01 08:23:27 [autobid.de] INFO: Scrape type for 3365060: 1
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366955%27 HTTP/1.1" 200 None
2026-04-01 08:23:27 [autobid.de] INFO: Scrape type for 3366955: 1
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365061%27 HTTP/1.1" 200 None
2026-04-01 08:23:27 [autobid.de] INFO: Scrape type for 3365061: 1
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366956%27 HTTP/1.1" 200 None
2026-04-01 08:23:27 [autobid.de] INFO: Scrape type for 3366956: 1
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365062%27 HTTP/1.1" 200 None
2026-04-01 08:23:27 [autobid.de] INFO: Scrape type for 3365062: 1
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366957%27 HTTP/1.1" 200 None
2026-04-01 08:23:27 [autobid.de] INFO: Scrape type for 3366957: 1
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365149%27 HTTP/1.1" 200 None
2026-04-01 08:23:27 [autobid.de] INFO: Scrape type for 3365149: 1
2026-04-01 08:24:07 [scrapy.extensions.logstats] INFO: Crawled 48 pages (at 7 pages/min), scraped 2 items (at 0 items/min)
2026-04-01 08:24:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 224MiB
2026-04-01 08:25:07 [scrapy.extensions.logstats] INFO: Crawled 48 pages (at 0 pages/min), scraped 2 items (at 0 items/min)
2026-04-01 08:25:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 224MiB
2026-04-01 08:26:07 [scrapy.extensions.logstats] INFO: Crawled 48 pages (at 0 pages/min), scraped 2 items (at 0 items/min)
2026-04-01 08:26:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 224MiB
2026-04-01 08:26:16 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 949471e8-ed42-5978-8cc1-d997eca79b31 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33600/3359994/93494552_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:26:18 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 949471e8-ed42-5978-8cc1-d997eca79b31 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33600/3359994/93494551_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:26:18 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mercedes-benz-gle-63-s-amg-4matic-automatik-3365149/details> (referer: None)
2026-04-01 08:26:18 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 1a8214e8-1739-51b2-b435-8952f55a5d7e with identifier: 3365149
2026-04-01 08:26:18 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 1a8214e8-1739-51b2-b435-8952f55a5d7e with scrape type 1
2026-04-01 08:26:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 1a8214e8-1739-51b2-b435-8952f55a5d7e
2026-04-01 08:26:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33652%252F3365149%252F93565066_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:26:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33652/3365149/93565066_hd.jpg not downloaded yet
2026-04-01 08:26:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33652%252F3365149%252F93564780_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:26:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33652/3365149/93564780_hd.jpg not downloaded yet
2026-04-01 08:26:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33652%252F3365149%252F93565055_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:26:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33652/3365149/93565055_hd.jpg not downloaded yet
2026-04-01 08:26:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33652%252F3365149%252F93564787_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:26:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33652/3365149/93564787_hd.jpg not downloaded yet
2026-04-01 08:26:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33652%252F3365149%252F93565069_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:26:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33652/3365149/93565069_hd.jpg not downloaded yet
2026-04-01 08:26:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=14> (referer: None)
2026-04-01 08:26:19 [autobid.de] INFO: Page 14/142: found 20 items
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366961%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [autobid.de] INFO: Scrape type for 3366961: 1
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365253%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [autobid.de] INFO: Scrape type for 3365253: 1
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366962%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [autobid.de] INFO: Scrape type for 3366962: 1
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365259%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [autobid.de] INFO: Scrape type for 3365259: 1
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366963%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [autobid.de] INFO: Scrape type for 3366963: 1
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365261%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [autobid.de] INFO: Scrape type for 3365261: 1
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366964%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [autobid.de] INFO: Scrape type for 3366964: 1
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365545%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [autobid.de] INFO: Scrape type for 3365545: 1
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366965%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [autobid.de] INFO: Scrape type for 3366965: 1
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365546%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [autobid.de] INFO: Scrape type for 3365546: 1
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366966%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [autobid.de] INFO: Scrape type for 3366966: 1
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365549%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [autobid.de] INFO: Scrape type for 3365549: 1
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366967%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [autobid.de] INFO: Scrape type for 3366967: 1
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365552%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [autobid.de] INFO: Scrape type for 3365552: 1
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366968%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [autobid.de] INFO: Scrape type for 3366968: 1
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365551%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [autobid.de] INFO: Scrape type for 3365551: 1
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366969%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [autobid.de] INFO: Scrape type for 3366969: 1
2026-04-01 08:26:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-sq7-competition-plus-4-0-tfsi-quattro-tiptronic-3366957/details> (referer: None)
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366375%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [autobid.de] INFO: Scrape type for 3366375: 1
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366970%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [autobid.de] INFO: Scrape type for 3366970: 1
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366377%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [autobid.de] INFO: Scrape type for 3366377: 1
2026-04-01 08:26:19 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: c4df6f36-9714-53cb-87cb-c47dfdae996d with identifier: 3366957
2026-04-01 08:26:19 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item c4df6f36-9714-53cb-87cb-c47dfdae996d with scrape type 1
2026-04-01 08:26:19 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item c4df6f36-9714-53cb-87cb-c47dfdae996d
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33670%252F3366957%252F93652287_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33670/3366957/93652287_hd.jpg not downloaded yet
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33670%252F3366957%252F93652289_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33670/3366957/93652289_hd.jpg not downloaded yet
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33670%252F3366957%252F93652301_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33670/3366957/93652301_hd.jpg not downloaded yet
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33670%252F3366957%252F93652316_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33670/3366957/93652316_hd.jpg not downloaded yet
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33670%252F3366957%252F93652303_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:26:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33670/3366957/93652303_hd.jpg not downloaded yet
2026-04-01 08:26:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item a851f583-7912-59fe-965f-2d6a0a222d86 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33647/3364683/93543688_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:26:21 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-q7-50-tdi-quattro-tiptronic-s-line-3366969/details> (referer: None)
2026-04-01 08:26:21 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: f92b7ccf-d103-51af-beb7-20e8787b87d5 with identifier: 3366969
2026-04-01 08:26:21 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f92b7ccf-d103-51af-beb7-20e8787b87d5 with scrape type 1
2026-04-01 08:26:21 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item f92b7ccf-d103-51af-beb7-20e8787b87d5
2026-04-01 08:26:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33670%252F3366969%252F93652684_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:26:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33670/3366969/93652684_hd.jpg not downloaded yet
2026-04-01 08:26:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33670%252F3366969%252F93652676_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:26:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33670/3366969/93652676_hd.jpg not downloaded yet
2026-04-01 08:26:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33670%252F3366969%252F93652687_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:26:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33670/3366969/93652687_hd.jpg not downloaded yet
2026-04-01 08:26:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33670%252F3366969%252F93652688_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:26:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33670/3366969/93652688_hd.jpg not downloaded yet
2026-04-01 08:26:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:26:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33670%252F3366969%252F93652685_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:26:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33670/3366969/93652685_hd.jpg not downloaded yet
2026-04-01 08:26:22 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 7c217fdb-b164-5b5d-8d21-ed7086eda7f5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33647/3364684/93543744_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:26:23 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 7c217fdb-b164-5b5d-8d21-ed7086eda7f5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33647/3364684/93543742_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:26:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 7c217fdb-b164-5b5d-8d21-ed7086eda7f5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33647/3364684/93543743_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:26:26 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 7c217fdb-b164-5b5d-8d21-ed7086eda7f5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33647/3364684/93543748_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:26:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item c7715c5a-8bd3-54eb-aaf5-b7dcf94eee20 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33606/3360553/93618604_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:27:07 [scrapy.extensions.logstats] INFO: Crawled 52 pages (at 4 pages/min), scraped 2 items (at 0 items/min)
2026-04-01 08:27:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 225MiB
2026-04-01 08:28:07 [scrapy.extensions.logstats] INFO: Crawled 52 pages (at 0 pages/min), scraped 2 items (at 0 items/min)
2026-04-01 08:28:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 225MiB
2026-04-01 08:29:07 [scrapy.extensions.logstats] INFO: Crawled 52 pages (at 0 pages/min), scraped 2 items (at 0 items/min)
2026-04-01 08:29:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 225MiB
2026-04-01 08:29:16 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 97ebdcf7-f035-5f0e-86d9-d477b0a86dca due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33591/3359084/93452767_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:29:18 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 97ebdcf7-f035-5f0e-86d9-d477b0a86dca due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33591/3359084/93452766_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:29:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item a783c9f0-5f13-5a04-bc4e-01698d0b1311 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33647/3364621/93541079_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:29:22 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item a783c9f0-5f13-5a04-bc4e-01698d0b1311 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33647/3364621/93541094_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:29:23 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item a783c9f0-5f13-5a04-bc4e-01698d0b1311 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33647/3364621/93541104_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:29:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mini-countryman-cooper-se-all4-hybrid-steptronic-3366377/details> (referer: None)
2026-04-01 08:29:24 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 8e4ffaf9-234c-575e-ab4e-61ef15361d06 with identifier: 3366377
2026-04-01 08:29:24 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 8e4ffaf9-234c-575e-ab4e-61ef15361d06 with scrape type 1
2026-04-01 08:29:24 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 8e4ffaf9-234c-575e-ab4e-61ef15361d06
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33664%252F3366377%252F93616586_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:29:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33664/3366377/93616586_hd.jpg not downloaded yet
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33664%252F3366377%252F93616593_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:29:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33664/3366377/93616593_hd.jpg not downloaded yet
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33664%252F3366377%252F93616576_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:29:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33664/3366377/93616576_hd.jpg not downloaded yet
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33664%252F3366377%252F93616573_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:29:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33664/3366377/93616573_hd.jpg not downloaded yet
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33664%252F3366377%252F93616589_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:29:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33664/3366377/93616589_hd.jpg not downloaded yet
2026-04-01 08:29:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=15> (referer: None)
2026-04-01 08:29:24 [autobid.de] INFO: Page 15/142: found 20 items
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364766%27 HTTP/1.1" 200 None
2026-04-01 08:29:24 [autobid.de] INFO: Scrape type for 3364766: 1
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354811%27 HTTP/1.1" 200 None
2026-04-01 08:29:24 [autobid.de] INFO: Scrape type for 3354811: 1
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365201%27 HTTP/1.1" 200 None
2026-04-01 08:29:24 [autobid.de] INFO: Scrape type for 3365201: 1
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365127%27 HTTP/1.1" 200 None
2026-04-01 08:29:24 [autobid.de] INFO: Scrape type for 3365127: 1
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365115%27 HTTP/1.1" 200 None
2026-04-01 08:29:24 [autobid.de] INFO: Scrape type for 3365115: 1
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365248%27 HTTP/1.1" 200 None
2026-04-01 08:29:24 [autobid.de] INFO: Scrape type for 3365248: 1
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365250%27 HTTP/1.1" 200 None
2026-04-01 08:29:24 [autobid.de] INFO: Scrape type for 3365250: 1
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365254%27 HTTP/1.1" 200 None
2026-04-01 08:29:24 [autobid.de] INFO: Scrape type for 3365254: 1
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365256%27 HTTP/1.1" 200 None
2026-04-01 08:29:24 [autobid.de] INFO: Scrape type for 3365256: 1
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365249%27 HTTP/1.1" 200 None
2026-04-01 08:29:24 [autobid.de] INFO: Scrape type for 3365249: 1
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365246%27 HTTP/1.1" 200 None
2026-04-01 08:29:24 [autobid.de] INFO: Scrape type for 3365246: 1
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365258%27 HTTP/1.1" 200 None
2026-04-01 08:29:24 [autobid.de] INFO: Scrape type for 3365258: 1
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365260%27 HTTP/1.1" 200 None
2026-04-01 08:29:24 [autobid.de] INFO: Scrape type for 3365260: 1
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365262%27 HTTP/1.1" 200 None
2026-04-01 08:29:25 [autobid.de] INFO: Scrape type for 3365262: 1
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365263%27 HTTP/1.1" 200 None
2026-04-01 08:29:25 [autobid.de] INFO: Scrape type for 3365263: 1
2026-04-01 08:29:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 479734c2-df31-5e26-a862-9b7d652bd054 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33648/3364741/93546410_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365537%27 HTTP/1.1" 200 None
2026-04-01 08:29:25 [autobid.de] INFO: Scrape type for 3365537: 1
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365538%27 HTTP/1.1" 200 None
2026-04-01 08:29:25 [autobid.de] INFO: Scrape type for 3365538: 1
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365257%27 HTTP/1.1" 200 None
2026-04-01 08:29:25 [autobid.de] INFO: Scrape type for 3365257: 1
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364767%27 HTTP/1.1" 200 None
2026-04-01 08:29:25 [autobid.de] INFO: Scrape type for 3364767: 1
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365547%27 HTTP/1.1" 200 None
2026-04-01 08:29:25 [autobid.de] INFO: Scrape type for 3365547: 1
2026-04-01 08:29:25 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-s6-avant-3-0-tdi-quattro-tiptronic-3366970/details> (referer: None)
2026-04-01 08:29:25 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 9098f66b-ef01-5969-8b85-3c8dbd75e2d0 with identifier: 3366970
2026-04-01 08:29:25 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9098f66b-ef01-5969-8b85-3c8dbd75e2d0 with scrape type 1
2026-04-01 08:29:25 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 9098f66b-ef01-5969-8b85-3c8dbd75e2d0
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33670%252F3366970%252F93652704_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:29:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33670/3366970/93652704_hd.jpg not downloaded yet
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33670%252F3366970%252F93652712_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:29:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33670/3366970/93652712_hd.jpg not downloaded yet
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33670%252F3366970%252F93652693_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:29:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33670/3366970/93652693_hd.jpg not downloaded yet
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33670%252F3366970%252F93652716_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:29:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33670/3366970/93652716_hd.jpg not downloaded yet
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33670%252F3366970%252F93652708_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:29:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33670/3366970/93652708_hd.jpg not downloaded yet
2026-04-01 08:29:26 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 479734c2-df31-5e26-a862-9b7d652bd054 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33648/3364741/93546414_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:29:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/fiat-panda-1-3-m-jet-4x4-rock-3365256/details> (referer: None)
2026-04-01 08:29:26 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 6a3b2121-c48e-52ec-af77-82a8778db2f2 with identifier: 3365256
2026-04-01 08:29:26 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6a3b2121-c48e-52ec-af77-82a8778db2f2 with scrape type 1
2026-04-01 08:29:26 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 6a3b2121-c48e-52ec-af77-82a8778db2f2
2026-04-01 08:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33653%252F3365256%252F93580033_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:29:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33653/3365256/93580033_hd.jpg not downloaded yet
2026-04-01 08:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33653%252F3365256%252F93580044_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:29:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33653/3365256/93580044_hd.jpg not downloaded yet
2026-04-01 08:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33653%252F3365256%252F93580038_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:29:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33653/3365256/93580038_hd.jpg not downloaded yet
2026-04-01 08:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33653%252F3365256%252F93580045_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:29:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33653/3365256/93580045_hd.jpg not downloaded yet
2026-04-01 08:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33653%252F3365256%252F93580027_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:29:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33653/3365256/93580027_hd.jpg not downloaded yet
2026-04-01 08:29:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 479734c2-df31-5e26-a862-9b7d652bd054 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33648/3364741/93546419_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:29:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=16> (referer: None)
2026-04-01 08:29:29 [autobid.de] INFO: Page 16/142: found 20 items
2026-04-01 08:29:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365548%27 HTTP/1.1" 200 None
2026-04-01 08:29:30 [autobid.de] INFO: Scrape type for 3365548: 1
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365550%27 HTTP/1.1" 200 None
2026-04-01 08:29:30 [autobid.de] INFO: Scrape type for 3365550: 1
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365553%27 HTTP/1.1" 200 None
2026-04-01 08:29:30 [autobid.de] INFO: Scrape type for 3365553: 1
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365554%27 HTTP/1.1" 200 None
2026-04-01 08:29:30 [autobid.de] INFO: Scrape type for 3365554: 1
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365555%27 HTTP/1.1" 200 None
2026-04-01 08:29:30 [autobid.de] INFO: Scrape type for 3365555: 1
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365556%27 HTTP/1.1" 200 None
2026-04-01 08:29:30 [autobid.de] INFO: Scrape type for 3365556: 1
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365558%27 HTTP/1.1" 200 None
2026-04-01 08:29:30 [autobid.de] INFO: Scrape type for 3365558: 1
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366373%27 HTTP/1.1" 200 None
2026-04-01 08:29:30 [autobid.de] INFO: Scrape type for 3366373: 1
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365559%27 HTTP/1.1" 200 None
2026-04-01 08:29:30 [autobid.de] INFO: Scrape type for 3365559: 1
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365199%27 HTTP/1.1" 200 None
2026-04-01 08:29:30 [autobid.de] INFO: Scrape type for 3365199: 1
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365088%27 HTTP/1.1" 200 None
2026-04-01 08:29:30 [autobid.de] INFO: Scrape type for 3365088: 1
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365247%27 HTTP/1.1" 200 None
2026-04-01 08:29:30 [autobid.de] INFO: Scrape type for 3365247: 1
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365264%27 HTTP/1.1" 200 None
2026-04-01 08:29:30 [autobid.de] INFO: Scrape type for 3365264: 1
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365126%27 HTTP/1.1" 200 None
2026-04-01 08:29:30 [autobid.de] INFO: Scrape type for 3365126: 1
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364644%27 HTTP/1.1" 200 None
2026-04-01 08:29:30 [autobid.de] INFO: Scrape type for 3364644: 1
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364736%27 HTTP/1.1" 200 None
2026-04-01 08:29:30 [autobid.de] INFO: Scrape type for 3364736: 1
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364746%27 HTTP/1.1" 200 None
2026-04-01 08:29:30 [autobid.de] INFO: Scrape type for 3364746: 1
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364737%27 HTTP/1.1" 200 None
2026-04-01 08:29:30 [autobid.de] INFO: Scrape type for 3364737: 1
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364735%27 HTTP/1.1" 200 None
2026-04-01 08:29:30 [autobid.de] INFO: Scrape type for 3364735: 1
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365539%27 HTTP/1.1" 200 None
2026-04-01 08:29:30 [autobid.de] INFO: Scrape type for 3365539: 1
2026-04-01 08:30:07 [scrapy.extensions.logstats] INFO: Crawled 57 pages (at 5 pages/min), scraped 2 items (at 0 items/min)
2026-04-01 08:30:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 226MiB
2026-04-01 08:31:07 [scrapy.extensions.logstats] INFO: Crawled 57 pages (at 0 pages/min), scraped 2 items (at 0 items/min)
2026-04-01 08:31:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 226MiB
2026-04-01 08:32:07 [scrapy.extensions.logstats] INFO: Crawled 57 pages (at 0 pages/min), scraped 2 items (at 0 items/min)
2026-04-01 08:32:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 226MiB
2026-04-01 08:32:16 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 1a8214e8-1739-51b2-b435-8952f55a5d7e due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33652/3365149/93565069_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:32:18 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item c4df6f36-9714-53cb-87cb-c47dfdae996d due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33670/3366957/93652303_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:32:18 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-tiguan-2-0-tsi-4motion-tiptronic-sport-style-3365539/details> (referer: None)
2026-04-01 08:32:18 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: d1f59db0-d6a6-5bbc-b554-f628af352192 with identifier: 3365539
2026-04-01 08:32:18 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item d1f59db0-d6a6-5bbc-b554-f628af352192 with scrape type 1
2026-04-01 08:32:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item d1f59db0-d6a6-5bbc-b554-f628af352192
2026-04-01 08:32:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365539%252F93582689_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:32:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365539/93582689_hd.jpg not downloaded yet
2026-04-01 08:32:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365539%252F93582691_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:32:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365539/93582691_hd.jpg not downloaded yet
2026-04-01 08:32:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365539%252F93582686_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:32:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365539/93582686_hd.jpg not downloaded yet
2026-04-01 08:32:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365539%252F93582696_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:32:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365539/93582696_hd.jpg not downloaded yet
2026-04-01 08:32:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365539%252F93582685_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:32:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365539/93582685_hd.jpg not downloaded yet
2026-04-01 08:32:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=17> (referer: None)
2026-04-01 08:32:19 [autobid.de] INFO: Page 17/142: found 20 items
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366396%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [autobid.de] INFO: Scrape type for 3366396: 1
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366506%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [autobid.de] INFO: Scrape type for 3366506: 1
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366507%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [autobid.de] INFO: Scrape type for 3366507: 1
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366508%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [autobid.de] INFO: Scrape type for 3366508: 1
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366505%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [autobid.de] INFO: Scrape type for 3366505: 1
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366509%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [autobid.de] INFO: Scrape type for 3366509: 1
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366504%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [autobid.de] INFO: Scrape type for 3366504: 1
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366510%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [autobid.de] INFO: Scrape type for 3366510: 1
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366503%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [autobid.de] INFO: Scrape type for 3366503: 1
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273339336%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [autobid.de] INFO: Scrape type for 3339336: 1
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360144%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [autobid.de] INFO: Scrape type for 3360144: 1
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359693%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [autobid.de] INFO: Scrape type for 3359693: 1
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360145%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [autobid.de] INFO: Scrape type for 3360145: 1
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359809%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [autobid.de] INFO: Scrape type for 3359809: 1
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360323%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [autobid.de] INFO: Scrape type for 3360323: 1
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363377%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [autobid.de] INFO: Scrape type for 3363377: 1
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360146%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [autobid.de] INFO: Scrape type for 3360146: 1
2026-04-01 08:32:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/bmw-320i-touring-steptronic-sport-line-3364735/details> (referer: None)
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273350585%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [autobid.de] INFO: Scrape type for 3350585: 1
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354911%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [autobid.de] INFO: Scrape type for 3354911: 1
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273344339%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [autobid.de] INFO: Scrape type for 3344339: 1
2026-04-01 08:32:19 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 11b213ae-71ce-550a-ba6d-653db1f79f3b with identifier: 3364735
2026-04-01 08:32:19 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 11b213ae-71ce-550a-ba6d-653db1f79f3b with scrape type 1
2026-04-01 08:32:19 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 11b213ae-71ce-550a-ba6d-653db1f79f3b
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33648%252F3364735%252F93546181_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33648/3364735/93546181_hd.jpg not downloaded yet
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33648%252F3364735%252F93546166_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33648/3364735/93546166_hd.jpg not downloaded yet
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33648%252F3364735%252F93546167_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33648/3364735/93546167_hd.jpg not downloaded yet
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33648%252F3364735%252F93546168_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33648/3364735/93546168_hd.jpg not downloaded yet
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33648%252F3364735%252F93546182_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:32:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33648/3364735/93546182_hd.jpg not downloaded yet
2026-04-01 08:32:20 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/land-rover-range-rover-3-0-drive-select-autobiography-3363377/details> (referer: None)
2026-04-01 08:32:20 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 5aad6bd0-6966-56b7-82a1-7b9e16fc7642 with identifier: 3363377
2026-04-01 08:32:20 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 5aad6bd0-6966-56b7-82a1-7b9e16fc7642 with scrape type 1
2026-04-01 08:32:20 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 3 photos for item 5aad6bd0-6966-56b7-82a1-7b9e16fc7642
2026-04-01 08:32:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33634%252F3363377%252F93503982_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:32:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33634/3363377/93503982_hd.jpg not downloaded yet
2026-04-01 08:32:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33634%252F3363377%252F93503983_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:32:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33634/3363377/93503983_hd.jpg not downloaded yet
2026-04-01 08:32:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33634%252F3363377%252F93503984_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:32:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33634/3363377/93503984_hd.jpg not downloaded yet
2026-04-01 08:32:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item c4df6f36-9714-53cb-87cb-c47dfdae996d due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33670/3366957/93652316_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:32:22 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item f92b7ccf-d103-51af-beb7-20e8787b87d5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33670/3366969/93652685_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:32:23 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item f92b7ccf-d103-51af-beb7-20e8787b87d5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33670/3366969/93652688_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:32:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item f92b7ccf-d103-51af-beb7-20e8787b87d5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33670/3366969/93652687_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:32:26 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item f92b7ccf-d103-51af-beb7-20e8787b87d5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33670/3366969/93652676_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:32:27 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: f92b7ccf-d103-51af-beb7-20e8787b87d5 with identifier: 3366969
2026-04-01 08:32:27 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: f92b7ccf-d103-51af-beb7-20e8787b87d5 with identifier: 3366969
2026-04-01 08:32:28 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: f92b7ccf-d103-51af-beb7-20e8787b87d5 with identifier: 3366969 to the API
2026-04-01 08:32:28 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 08:32:28 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 08:32:28 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33670/3366969/93652684_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 81355',
 'body_type': 'All-terrain vehicle',
 'category': 'All-terrain vehicle',
 'color': 'white',
 'currency': 'EUR',
 'engine': '3.0L',
 'engine_horse_power': 286,
 'expiration_date': '2026-04-01T09:30:00Z',
 'fuel_type': 'Diesel',
 'id': 'f92b7ccf-d103-51af-beb7-20e8787b87d5',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_included': True,
 'link_to_web_offer': 'https://autobid.de/en/item/audi-q7-50-tdi-quattro-tiptronic-s-line-3366969',
 'listing': 'auction',
 'location': 'Weiterstadt',
 'make': 'Audi',
 'model': 'Q7',
 'odometer': 12800,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 5,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33670/3366969/93652684_hd.jpg',
                'https://cdn.autobid.de/data/cars/33670/3366969/93652676_hd.jpg',
                'https://cdn.autobid.de/data/cars/33670/3366969/93652687_hd.jpg',
                'https://cdn.autobid.de/data/cars/33670/3366969/93652688_hd.jpg',
                'https://cdn.autobid.de/data/cars/33670/3366969/93652685_hd.jpg'],
 'photos': [],
 'price': 64900,
 'price_includes_vat': True,
 'registration_date': '2025-02-01',
 'seats_number': 5,
 'seller_name': 'Audi Frankfurt GmbH',
 'title': 'Audi Q7 50 TDI quattro Tiptronic "S-line"',
 'transmission': 'automatic',
 'vin': None,
 'year': 2025}
2026-04-01 08:32:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366969%27 HTTP/1.1" 200 None
2026-04-01 08:32:28 [autobid.de] INFO: Saving data for 3366969: {'created_time': 1775032348.548372, 'last_price_update_time': 1775032348.548388, 'auction_closing_time': 1775035800.0}
2026-04-01 08:32:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:32:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3366969') HTTP/1.1" 204 0
2026-04-01 08:32:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item f92b7ccf-d103-51af-beb7-20e8787b87d5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33670/3366969/93652684_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:33:07 [scrapy.extensions.logstats] INFO: Crawled 61 pages (at 4 pages/min), scraped 3 items (at 1 items/min)
2026-04-01 08:33:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 229MiB
2026-04-01 08:34:07 [scrapy.extensions.logstats] INFO: Crawled 61 pages (at 0 pages/min), scraped 3 items (at 0 items/min)
2026-04-01 08:34:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 229MiB
2026-04-01 08:35:07 [scrapy.extensions.logstats] INFO: Crawled 61 pages (at 0 pages/min), scraped 3 items (at 0 items/min)
2026-04-01 08:35:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 229MiB
2026-04-01 08:35:16 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item c4df6f36-9714-53cb-87cb-c47dfdae996d due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33670/3366957/93652301_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:35:18 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item c4df6f36-9714-53cb-87cb-c47dfdae996d due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33670/3366957/93652289_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:35:20 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=18> (referer: None)
2026-04-01 08:35:20 [autobid.de] INFO: Page 18/142: found 20 items
2026-04-01 08:35:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359979%27 HTTP/1.1" 200 None
2026-04-01 08:35:20 [autobid.de] INFO: Scrape type for 3359979: 1
2026-04-01 08:35:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359801%27 HTTP/1.1" 200 None
2026-04-01 08:35:20 [autobid.de] INFO: Scrape type for 3359801: 1
2026-04-01 08:35:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359805%27 HTTP/1.1" 200 None
2026-04-01 08:35:20 [autobid.de] INFO: Scrape type for 3359805: 1
2026-04-01 08:35:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359797%27 HTTP/1.1" 200 None
2026-04-01 08:35:20 [autobid.de] INFO: Scrape type for 3359797: 1
2026-04-01 08:35:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359813%27 HTTP/1.1" 200 None
2026-04-01 08:35:20 [autobid.de] INFO: Scrape type for 3359813: 1
2026-04-01 08:35:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359795%27 HTTP/1.1" 200 None
2026-04-01 08:35:20 [autobid.de] INFO: Scrape type for 3359795: 1
2026-04-01 08:35:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359978%27 HTTP/1.1" 200 None
2026-04-01 08:35:20 [autobid.de] INFO: Scrape type for 3359978: 1
2026-04-01 08:35:20 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: c4df6f36-9714-53cb-87cb-c47dfdae996d with identifier: 3366957
2026-04-01 08:35:20 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: c4df6f36-9714-53cb-87cb-c47dfdae996d with identifier: 3366957
2026-04-01 08:35:21 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: c4df6f36-9714-53cb-87cb-c47dfdae996d with identifier: 3366957 to the API
2026-04-01 08:35:21 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 08:35:21 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 08:35:22 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33670/3366957/93652287_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 81355',
 'body_type': 'All-terrain vehicle',
 'category': 'All-terrain vehicle',
 'color': 'black Metallic',
 'currency': 'EUR',
 'emission_standard': 'EURO 6d',
 'engine': '4.0L',
 'engine_horse_power': 507,
 'expiration_date': '2026-04-01T09:30:00Z',
 'fuel_type': 'Petrol',
 'id': 'c4df6f36-9714-53cb-87cb-c47dfdae996d',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_included': True,
 'link_to_web_offer': 'https://autobid.de/en/item/audi-sq7-competition-plus-4-0-tfsi-quattro-tiptronic-3366957',
 'listing': 'auction',
 'location': 'Weiterstadt',
 'make': 'Audi',
 'model': 'SQ7 competition plus',
 'odometer': 85400,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 5,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33670/3366957/93652287_hd.jpg',
                'https://cdn.autobid.de/data/cars/33670/3366957/93652289_hd.jpg',
                'https://cdn.autobid.de/data/cars/33670/3366957/93652301_hd.jpg',
                'https://cdn.autobid.de/data/cars/33670/3366957/93652316_hd.jpg',
                'https://cdn.autobid.de/data/cars/33670/3366957/93652303_hd.jpg'],
 'photos': [],
 'price': 59900,
 'price_includes_vat': True,
 'registration_date': '2022-03-01',
 'seats_number': 7,
 'seller_name': 'Audi Frankfurt GmbH',
 'title': 'Audi SQ7 competition plus 4,0 TFSI quattro Tiptronic',
 'transmission': 'automatic',
 'vin': None,
 'year': 2022}
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366957%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [autobid.de] INFO: Saving data for 3366957: {'created_time': 1775032522.086822, 'last_price_update_time': 1775032522.086837, 'auction_closing_time': 1775035800.0}
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3366957') HTTP/1.1" 204 0
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359811%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [autobid.de] INFO: Scrape type for 3359811: 1
2026-04-01 08:35:22 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item c4df6f36-9714-53cb-87cb-c47dfdae996d due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33670/3366957/93652287_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:35:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mercedes-benz-sprinter-koffer-317-cdi-9g-tronic-3344339/details> (referer: None)
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345105%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [autobid.de] INFO: Scrape type for 3345105: 1
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345106%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [autobid.de] INFO: Scrape type for 3345106: 1
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345107%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [autobid.de] INFO: Scrape type for 3345107: 1
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359799%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [autobid.de] INFO: Scrape type for 3359799: 1
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359807%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [autobid.de] INFO: Scrape type for 3359807: 1
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359815%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [autobid.de] INFO: Scrape type for 3359815: 1
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345113%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [autobid.de] INFO: Scrape type for 3345113: 1
2026-04-01 08:35:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 3a7e8f60-9321-57c8-ab47-bd60df2b7c20 with identifier: 3344339
2026-04-01 08:35:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 3a7e8f60-9321-57c8-ab47-bd60df2b7c20 with scrape type 1
2026-04-01 08:35:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 3a7e8f60-9321-57c8-ab47-bd60df2b7c20
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33444%252F3344339%252F93490075_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33444/3344339/93490075_hd.jpg not downloaded yet
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345114%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [autobid.de] INFO: Scrape type for 3345114: 1
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33444%252F3344339%252F93490081_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33444/3344339/93490081_hd.jpg not downloaded yet
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345120%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [autobid.de] INFO: Scrape type for 3345120: 1
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33444%252F3344339%252F93490108_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33444/3344339/93490108_hd.jpg not downloaded yet
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345122%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [autobid.de] INFO: Scrape type for 3345122: 1
2026-04-01 08:35:22 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 8e4ffaf9-234c-575e-ab4e-61ef15361d06 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33664/3366377/93616589_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33444%252F3344339%252F93490102_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33444/3344339/93490102_hd.jpg not downloaded yet
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345124%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [autobid.de] INFO: Scrape type for 3345124: 1
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33444%252F3344339%252F93490120_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33444/3344339/93490120_hd.jpg not downloaded yet
2026-04-01 08:35:22 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 5aad6bd0-6966-56b7-82a1-7b9e16fc7642 due to An error occurred while connecting: 104: Connection reset by peer.
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345127%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [autobid.de] INFO: Scrape type for 3345127: 1
2026-04-01 08:35:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mercedes-benz-glc-250-cgi-coupe-4matic-9g-tronic-amg-line-3359979/details> (referer: None)
2026-04-01 08:35:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: f1844fb5-d886-5253-9755-1c9c1b2c5cd6 with identifier: 3359979
2026-04-01 08:35:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f1844fb5-d886-5253-9755-1c9c1b2c5cd6 with scrape type 1
2026-04-01 08:35:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item f1844fb5-d886-5253-9755-1c9c1b2c5cd6
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33600%252F3359979%252F93452692_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33600/3359979/93452692_hd.jpg not downloaded yet
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33600%252F3359979%252F93452693_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33600/3359979/93452693_hd.jpg not downloaded yet
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33600%252F3359979%252F93452694_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33600/3359979/93452694_hd.jpg not downloaded yet
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33600%252F3359979%252F93452695_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33600/3359979/93452695_hd.jpg not downloaded yet
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33600%252F3359979%252F93452696_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33600/3359979/93452696_hd.jpg not downloaded yet
2026-04-01 08:35:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-passat-variant-2-0-tdi-bmt-dsg-highline-3359811/details> (referer: None)
2026-04-01 08:35:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 57f1c9d7-eca8-5d13-aef9-9a57a0b96a96 with identifier: 3359811
2026-04-01 08:35:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 57f1c9d7-eca8-5d13-aef9-9a57a0b96a96 with scrape type 1
2026-04-01 08:35:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 57f1c9d7-eca8-5d13-aef9-9a57a0b96a96
2026-04-01 08:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33599%252F3359811%252F93459257_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33599/3359811/93459257_hd.jpg not downloaded yet
2026-04-01 08:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33599%252F3359811%252F93459258_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33599/3359811/93459258_hd.jpg not downloaded yet
2026-04-01 08:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33599%252F3359811%252F93459259_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33599/3359811/93459259_hd.jpg not downloaded yet
2026-04-01 08:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33599%252F3359811%252F93459260_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33599/3359811/93459260_hd.jpg not downloaded yet
2026-04-01 08:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33599%252F3359811%252F93459261_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33599/3359811/93459261_hd.jpg not downloaded yet
2026-04-01 08:35:23 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 9098f66b-ef01-5969-8b85-3c8dbd75e2d0 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33670/3366970/93652708_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:35:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mercedes-benz-marco-polo-250-d-rwd-9g-tronic-3345113/details> (referer: None)
2026-04-01 08:35:25 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 7583a479-884b-5d5f-8f71-44f1d304b5b6 with identifier: 3345113
2026-04-01 08:35:25 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 7583a479-884b-5d5f-8f71-44f1d304b5b6 with scrape type 1
2026-04-01 08:35:25 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 7583a479-884b-5d5f-8f71-44f1d304b5b6
2026-04-01 08:35:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33452%252F3345113%252F93441785_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33452/3345113/93441785_hd.jpg not downloaded yet
2026-04-01 08:35:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33452%252F3345113%252F93441793_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33452/3345113/93441793_hd.jpg not downloaded yet
2026-04-01 08:35:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33452%252F3345113%252F93441788_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33452/3345113/93441788_hd.jpg not downloaded yet
2026-04-01 08:35:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33452%252F3345113%252F93441792_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33452/3345113/93441792_hd.jpg not downloaded yet
2026-04-01 08:35:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33452%252F3345113%252F93441782_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33452/3345113/93441782_hd.jpg not downloaded yet
2026-04-01 08:35:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 9098f66b-ef01-5969-8b85-3c8dbd75e2d0 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33670/3366970/93652716_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:35:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mercedes-benz-marco-polo-220-d-rwd-9g-tronic-3345114/details> (referer: None)
2026-04-01 08:35:26 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 6df12d10-efdc-5862-8caa-04e6bf1ff949 with identifier: 3345114
2026-04-01 08:35:26 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6df12d10-efdc-5862-8caa-04e6bf1ff949 with scrape type 1
2026-04-01 08:35:26 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 6df12d10-efdc-5862-8caa-04e6bf1ff949
2026-04-01 08:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33452%252F3345114%252F93396939_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33452/3345114/93396939_hd.jpg not downloaded yet
2026-04-01 08:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33452%252F3345114%252F93396936_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33452/3345114/93396936_hd.jpg not downloaded yet
2026-04-01 08:35:26 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 6a3b2121-c48e-52ec-af77-82a8778db2f2 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33653/3365256/93580027_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33452%252F3345114%252F93396915_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33452/3345114/93396915_hd.jpg not downloaded yet
2026-04-01 08:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33452%252F3345114%252F93396929_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33452/3345114/93396929_hd.jpg not downloaded yet
2026-04-01 08:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33452%252F3345114%252F93396932_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:35:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33452/3345114/93396932_hd.jpg not downloaded yet
2026-04-01 08:35:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 6a3b2121-c48e-52ec-af77-82a8778db2f2 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33653/3365256/93580045_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:35:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=19> (referer: None)
2026-04-01 08:35:29 [autobid.de] INFO: Page 19/142: found 20 items
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361400%27 HTTP/1.1" 200 None
2026-04-01 08:35:29 [autobid.de] INFO: Scrape type for 3361400: 1
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363191%27 HTTP/1.1" 200 None
2026-04-01 08:35:29 [autobid.de] INFO: Scrape type for 3363191: 1
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357502%27 HTTP/1.1" 200 None
2026-04-01 08:35:29 [autobid.de] INFO: Scrape type for 3357502: 1
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360131%27 HTTP/1.1" 200 None
2026-04-01 08:35:29 [autobid.de] INFO: Scrape type for 3360131: 1
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355202%27 HTTP/1.1" 200 None
2026-04-01 08:35:29 [autobid.de] INFO: Scrape type for 3355202: 1
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358327%27 HTTP/1.1" 200 None
2026-04-01 08:35:29 [autobid.de] INFO: Scrape type for 3358327: 1
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355472%27 HTTP/1.1" 200 None
2026-04-01 08:35:29 [autobid.de] INFO: Scrape type for 3355472: 1
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360485%27 HTTP/1.1" 200 None
2026-04-01 08:35:29 [autobid.de] INFO: Scrape type for 3360485: 1
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355189%27 HTTP/1.1" 200 None
2026-04-01 08:35:29 [autobid.de] INFO: Scrape type for 3355189: 1
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360357%27 HTTP/1.1" 200 None
2026-04-01 08:35:29 [autobid.de] INFO: Scrape type for 3360357: 1
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355207%27 HTTP/1.1" 200 None
2026-04-01 08:35:29 [autobid.de] INFO: Scrape type for 3355207: 1
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358328%27 HTTP/1.1" 200 None
2026-04-01 08:35:29 [autobid.de] INFO: Scrape type for 3358328: 1
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360356%27 HTTP/1.1" 200 None
2026-04-01 08:35:29 [autobid.de] INFO: Scrape type for 3360356: 1
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355213%27 HTTP/1.1" 200 None
2026-04-01 08:35:29 [autobid.de] INFO: Scrape type for 3355213: 1
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360486%27 HTTP/1.1" 200 None
2026-04-01 08:35:29 [autobid.de] INFO: Scrape type for 3360486: 1
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355210%27 HTTP/1.1" 200 None
2026-04-01 08:35:29 [autobid.de] INFO: Scrape type for 3355210: 1
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360361%27 HTTP/1.1" 200 None
2026-04-01 08:35:29 [autobid.de] INFO: Scrape type for 3360361: 1
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355214%27 HTTP/1.1" 200 None
2026-04-01 08:35:29 [autobid.de] INFO: Scrape type for 3355214: 1
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360113%27 HTTP/1.1" 200 None
2026-04-01 08:35:29 [autobid.de] INFO: Scrape type for 3360113: 1
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355203%27 HTTP/1.1" 200 None
2026-04-01 08:35:29 [autobid.de] INFO: Scrape type for 3355203: 1
2026-04-01 08:36:07 [scrapy.extensions.logstats] INFO: Crawled 68 pages (at 7 pages/min), scraped 4 items (at 1 items/min)
2026-04-01 08:36:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 231MiB
2026-04-01 08:37:07 [scrapy.extensions.logstats] INFO: Crawled 68 pages (at 0 pages/min), scraped 4 items (at 0 items/min)
2026-04-01 08:37:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 231MiB
2026-04-01 08:38:07 [scrapy.extensions.logstats] INFO: Crawled 68 pages (at 0 pages/min), scraped 4 items (at 0 items/min)
2026-04-01 08:38:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 231MiB
2026-04-01 08:38:16 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item d1f59db0-d6a6-5bbc-b554-f628af352192 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33656/3365539/93582685_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:38:18 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 11b213ae-71ce-550a-ba6d-653db1f79f3b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33648/3364735/93546182_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:38:18 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-q5-50-tfsi-e-quattro-s-tronic-advanced-3355203/details> (referer: None)
2026-04-01 08:38:18 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 19c99c22-d350-53e3-ae43-7dc0c6dc4dde with identifier: 3355203
2026-04-01 08:38:18 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 19c99c22-d350-53e3-ae43-7dc0c6dc4dde with scrape type 1
2026-04-01 08:38:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 19c99c22-d350-53e3-ae43-7dc0c6dc4dde
2026-04-01 08:38:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33553%252F3355203%252F93368345_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:38:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33553/3355203/93368345_hd.jpg not downloaded yet
2026-04-01 08:38:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33553%252F3355203%252F93368346_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:38:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33553/3355203/93368346_hd.jpg not downloaded yet
2026-04-01 08:38:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33553%252F3355203%252F93368347_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:38:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33553/3355203/93368347_hd.jpg not downloaded yet
2026-04-01 08:38:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33553%252F3355203%252F93368348_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:38:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33553/3355203/93368348_hd.jpg not downloaded yet
2026-04-01 08:38:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33553%252F3355203%252F93368349_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:38:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33553/3355203/93368349_hd.jpg not downloaded yet
2026-04-01 08:38:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=20> (referer: None)
2026-04-01 08:38:19 [autobid.de] INFO: Page 20/142: found 20 items
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355192%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [autobid.de] INFO: Scrape type for 3355192: 1
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358326%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [autobid.de] INFO: Scrape type for 3358326: 1
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355209%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [autobid.de] INFO: Scrape type for 3355209: 1
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360114%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [autobid.de] INFO: Scrape type for 3360114: 1
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355198%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [autobid.de] INFO: Scrape type for 3355198: 1
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360132%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [autobid.de] INFO: Scrape type for 3360132: 1
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355206%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [autobid.de] INFO: Scrape type for 3355206: 1
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360355%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [autobid.de] INFO: Scrape type for 3360355: 1
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355204%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [autobid.de] INFO: Scrape type for 3355204: 1
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360362%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [autobid.de] INFO: Scrape type for 3360362: 1
2026-04-01 08:38:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-amarok-dc-3-0-tdi-4motion-tiptronic-dark-label-3360113/details> (referer: None)
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355199%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [autobid.de] INFO: Scrape type for 3355199: 1
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358667%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [autobid.de] INFO: Scrape type for 3358667: 1
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355211%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [autobid.de] INFO: Scrape type for 3355211: 1
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360358%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [autobid.de] INFO: Scrape type for 3360358: 1
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355191%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [autobid.de] INFO: Scrape type for 3355191: 1
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360359%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [autobid.de] INFO: Scrape type for 3360359: 1
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355194%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [autobid.de] INFO: Scrape type for 3355194: 1
2026-04-01 08:38:19 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: ad1ff5f5-7d8d-552d-b22b-4ecdf1d28d85 with identifier: 3360113
2026-04-01 08:38:19 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ad1ff5f5-7d8d-552d-b22b-4ecdf1d28d85 with scrape type 1
2026-04-01 08:38:19 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item ad1ff5f5-7d8d-552d-b22b-4ecdf1d28d85
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33602%252F3360113%252F93511459_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33602/3360113/93511459_hd.jpg not downloaded yet
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358330%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [autobid.de] INFO: Scrape type for 3358330: 1
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33602%252F3360113%252F93511464_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33602/3360113/93511464_hd.jpg not downloaded yet
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355197%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [autobid.de] INFO: Scrape type for 3355197: 1
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33602%252F3360113%252F93511469_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33602/3360113/93511469_hd.jpg not downloaded yet
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360133%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [autobid.de] INFO: Scrape type for 3360133: 1
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33602%252F3360113%252F93511474_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33602/3360113/93511474_hd.jpg not downloaded yet
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33602%252F3360113%252F93511479_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:38:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33602/3360113/93511479_hd.jpg not downloaded yet
2026-04-01 08:38:20 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-t4-multivan-2-8-lpg-automatik-trendline-3360362/details> (referer: None)
2026-04-01 08:38:20 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 02b5fab7-15ba-5e78-b798-a9e0fc0c92fc with identifier: 3360362
2026-04-01 08:38:20 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 02b5fab7-15ba-5e78-b798-a9e0fc0c92fc with scrape type 1
2026-04-01 08:38:20 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 02b5fab7-15ba-5e78-b798-a9e0fc0c92fc
2026-04-01 08:38:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33604%252F3360362%252F93407023_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:38:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33604/3360362/93407023_hd.jpg not downloaded yet
2026-04-01 08:38:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33604%252F3360362%252F93407026_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:38:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33604/3360362/93407026_hd.jpg not downloaded yet
2026-04-01 08:38:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33604%252F3360362%252F93407022_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:38:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33604/3360362/93407022_hd.jpg not downloaded yet
2026-04-01 08:38:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33604%252F3360362%252F93407017_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:38:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33604/3360362/93407017_hd.jpg not downloaded yet
2026-04-01 08:38:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33604%252F3360362%252F93407028_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:38:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33604/3360362/93407028_hd.jpg not downloaded yet
2026-04-01 08:38:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 5aad6bd0-6966-56b7-82a1-7b9e16fc7642 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33634/3363377/93503984_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:38:23 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 5aad6bd0-6966-56b7-82a1-7b9e16fc7642 with identifier: 3363377
2026-04-01 08:38:23 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 08:38:23 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=grey HTTP/1.1" 200 None
2026-04-01 08:38:23 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 08:38:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 08:38:23 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 08:38:23 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=Hybrid HTTP/1.1" 200 None
2026-04-01 08:38:23 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 08:38:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 08:38:23 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 5aad6bd0-6966-56b7-82a1-7b9e16fc7642 with identifier: 3363377
2026-04-01 08:38:24 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: 5aad6bd0-6966-56b7-82a1-7b9e16fc7642 with identifier: 3363377 to the API
2026-04-01 08:38:24 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 08:38:24 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 08:38:24 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33634/3363377/93503982_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 81294',
 'body_type': 'All-terrain vehicle',
 'category': 'All-terrain vehicle',
 'color': 'grey',
 'currency': 'EUR',
 'engine': '3.0L',
 'engine_horse_power': 360,
 'expiration_date': '2026-04-01T10:00:00Z',
 'fuel_type': 'Hybrid',
 'id': '5aad6bd0-6966-56b7-82a1-7b9e16fc7642',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'link_to_web_offer': 'https://autobid.de/en/item/land-rover-range-rover-3-0-drive-select-autobiography-3363377',
 'listing': 'auction',
 'location': 'Marl',
 'make': 'Land Rover',
 'model': 'Range Rover',
 'odometer': 3200,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 3,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33634/3363377/93503982_hd.jpg',
                'https://cdn.autobid.de/data/cars/33634/3363377/93503983_hd.jpg',
                'https://cdn.autobid.de/data/cars/33634/3363377/93503984_hd.jpg'],
 'photos': [],
 'price': 94100,
 'price_includes_vat': False,
 'registration_date': '2025-07-01',
 'seller_name': 'Subsidiary/authorised dealer of the brand(s): VW',
 'title': 'Land Rover Range Rover 3,0 Drive Select "Autobiography"',
 'transmission': 'automatic',
 'vin': None,
 'year': 2025}
2026-04-01 08:38:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363377%27 HTTP/1.1" 200 None
2026-04-01 08:38:24 [autobid.de] INFO: Saving data for 3363377: {'created_time': 1775032704.287832, 'last_price_update_time': 1775032704.287844, 'auction_closing_time': 1775037600.0}
2026-04-01 08:38:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3363377') HTTP/1.1" 204 0
2026-04-01 08:38:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 5aad6bd0-6966-56b7-82a1-7b9e16fc7642 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33634/3363377/93503982_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:38:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 11b213ae-71ce-550a-ba6d-653db1f79f3b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33648/3364735/93546168_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:38:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 11b213ae-71ce-550a-ba6d-653db1f79f3b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33648/3364735/93546167_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:38:26 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 11b213ae-71ce-550a-ba6d-653db1f79f3b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33648/3364735/93546166_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:38:28 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 11b213ae-71ce-550a-ba6d-653db1f79f3b with identifier: 3364735
2026-04-01 08:38:28 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 11b213ae-71ce-550a-ba6d-653db1f79f3b with identifier: 3364735
2026-04-01 08:38:28 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: 11b213ae-71ce-550a-ba6d-653db1f79f3b with identifier: 3364735 to the API
2026-04-01 08:38:28 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 08:38:28 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 08:38:28 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33648/3364735/93546181_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 81314',
 'body_type': 'Estate',
 'category': 'Estate',
 'color': 'red Metallic',
 'currency': 'EUR',
 'emission_standard': 'EURO 6',
 'engine': '2.0L',
 'engine_horse_power': 184,
 'expiration_date': '2026-04-01T09:30:00Z',
 'fuel_type': 'Petrol',
 'id': '11b213ae-71ce-550a-ba6d-653db1f79f3b',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'link_to_web_offer': 'https://autobid.de/en/item/bmw-320i-touring-steptronic-sport-line-3364735',
 'listing': 'auction',
 'location': 'Emmering',
 'make': 'BMW',
 'model': '320i',
 'odometer': 188200,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 5,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33648/3364735/93546181_hd.jpg',
                'https://cdn.autobid.de/data/cars/33648/3364735/93546166_hd.jpg',
                'https://cdn.autobid.de/data/cars/33648/3364735/93546167_hd.jpg',
                'https://cdn.autobid.de/data/cars/33648/3364735/93546168_hd.jpg',
                'https://cdn.autobid.de/data/cars/33648/3364735/93546182_hd.jpg'],
 'photos': [],
 'price': 8400,
 'price_includes_vat': False,
 'registration_date': '2013-07-01',
 'seats_number': 5,
 'seller_name': 'Subsidiary/authorised dealer of the brand(s): Mini, BMW',
 'title': 'BMW 320i Touring Steptronic "Sport Line"',
 'transmission': 'automatic',
 'vin': None,
 'year': 2013}
2026-04-01 08:38:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364735%27 HTTP/1.1" 200 None
2026-04-01 08:38:28 [autobid.de] INFO: Saving data for 3364735: {'created_time': 1775032708.732001, 'last_price_update_time': 1775032708.732014, 'auction_closing_time': 1775035800.0}
2026-04-01 08:38:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3364735') HTTP/1.1" 204 0
2026-04-01 08:38:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 11b213ae-71ce-550a-ba6d-653db1f79f3b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33648/3364735/93546181_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:38:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=21> (referer: None)
2026-04-01 08:38:30 [autobid.de] INFO: Page 21/142: found 20 items
2026-04-01 08:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355195%27 HTTP/1.1" 200 None
2026-04-01 08:38:30 [autobid.de] INFO: Scrape type for 3355195: 1
2026-04-01 08:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358668%27 HTTP/1.1" 200 None
2026-04-01 08:38:30 [autobid.de] INFO: Scrape type for 3358668: 1
2026-04-01 08:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355200%27 HTTP/1.1" 200 None
2026-04-01 08:38:30 [autobid.de] INFO: Scrape type for 3355200: 1
2026-04-01 08:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353949%27 HTTP/1.1" 200 None
2026-04-01 08:38:30 [autobid.de] INFO: Scrape type for 3353949: 1
2026-04-01 08:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362797%27 HTTP/1.1" 200 None
2026-04-01 08:38:30 [autobid.de] INFO: Scrape type for 3362797: 1
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358970%27 HTTP/1.1" 200 None
2026-04-01 08:38:31 [autobid.de] INFO: Scrape type for 3358970: 1
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362625%27 HTTP/1.1" 200 None
2026-04-01 08:38:31 [autobid.de] INFO: Scrape type for 3362625: 1
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358971%27 HTTP/1.1" 200 None
2026-04-01 08:38:31 [autobid.de] INFO: Scrape type for 3358971: 1
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362786%27 HTTP/1.1" 200 None
2026-04-01 08:38:31 [autobid.de] INFO: Scrape type for 3362786: 1
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362626%27 HTTP/1.1" 200 None
2026-04-01 08:38:31 [autobid.de] INFO: Scrape type for 3362626: 1
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358973%27 HTTP/1.1" 200 None
2026-04-01 08:38:31 [autobid.de] INFO: Scrape type for 3358973: 1
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362792%27 HTTP/1.1" 200 None
2026-04-01 08:38:31 [autobid.de] INFO: Scrape type for 3362792: 1
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358974%27 HTTP/1.1" 200 None
2026-04-01 08:38:31 [autobid.de] INFO: Scrape type for 3358974: 1
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362624%27 HTTP/1.1" 200 None
2026-04-01 08:38:31 [autobid.de] INFO: Scrape type for 3362624: 1
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358975%27 HTTP/1.1" 200 None
2026-04-01 08:38:31 [autobid.de] INFO: Scrape type for 3358975: 1
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362783%27 HTTP/1.1" 200 None
2026-04-01 08:38:31 [autobid.de] INFO: Scrape type for 3362783: 1
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358976%27 HTTP/1.1" 200 None
2026-04-01 08:38:31 [autobid.de] INFO: Scrape type for 3358976: 1
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362622%27 HTTP/1.1" 200 None
2026-04-01 08:38:31 [autobid.de] INFO: Scrape type for 3362622: 1
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358977%27 HTTP/1.1" 200 None
2026-04-01 08:38:31 [autobid.de] INFO: Scrape type for 3358977: 1
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362798%27 HTTP/1.1" 200 None
2026-04-01 08:38:31 [autobid.de] INFO: Scrape type for 3362798: 1
2026-04-01 08:39:07 [scrapy.extensions.logstats] INFO: Crawled 73 pages (at 5 pages/min), scraped 6 items (at 2 items/min)
2026-04-01 08:39:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 231MiB
2026-04-01 08:40:07 [scrapy.extensions.logstats] INFO: Crawled 73 pages (at 0 pages/min), scraped 6 items (at 0 items/min)
2026-04-01 08:40:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 231MiB
2026-04-01 08:41:07 [scrapy.extensions.logstats] INFO: Crawled 73 pages (at 0 pages/min), scraped 6 items (at 0 items/min)
2026-04-01 08:41:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 231MiB
2026-04-01 08:41:16 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item f1844fb5-d886-5253-9755-1c9c1b2c5cd6 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33600/3359979/93452696_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:41:18 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 57f1c9d7-eca8-5d13-aef9-9a57a0b96a96 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33599/3359811/93459260_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:41:18 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/vw-caddy-2-0-tdi-dsg-style-3362798/details> (referer: None)
2026-04-01 08:41:18 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 08:41:18 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/vw-caddy-2-0-tdi-dsg-style-3362798/details> (referer: None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
    current.result = callback(  # type: ignore[misc]
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
    return self._process_spider_exception(response, spider, _failure)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
    result = method(response=response, exception=exception, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
    raise exception
  File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
    result = method(response=response, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
    raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 08:41:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=22> (referer: None)
2026-04-01 08:41:19 [autobid.de] INFO: Page 22/143: found 20 items
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358979%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [autobid.de] INFO: Scrape type for 3358979: 1
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362789%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [autobid.de] INFO: Scrape type for 3362789: 1
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358980%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [autobid.de] INFO: Scrape type for 3358980: 1
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362800%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [autobid.de] INFO: Scrape type for 3362800: 1
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358981%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [autobid.de] INFO: Scrape type for 3358981: 1
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362795%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [autobid.de] INFO: Scrape type for 3362795: 1
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358982%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [autobid.de] INFO: Scrape type for 3358982: 1
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362793%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [autobid.de] INFO: Scrape type for 3362793: 1
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358983%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [autobid.de] INFO: Scrape type for 3358983: 1
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362796%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [autobid.de] INFO: Scrape type for 3362796: 1
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358984%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [autobid.de] INFO: Scrape type for 3358984: 1
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362623%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [autobid.de] INFO: Scrape type for 3362623: 1
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358985%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [autobid.de] INFO: Scrape type for 3358985: 1
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362794%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [autobid.de] INFO: Scrape type for 3362794: 1
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358986%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [autobid.de] INFO: Scrape type for 3358986: 1
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362799%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [autobid.de] INFO: Scrape type for 3362799: 1
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358987%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [autobid.de] INFO: Scrape type for 3358987: 1
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362791%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [autobid.de] INFO: Scrape type for 3362791: 1
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358988%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [autobid.de] INFO: Scrape type for 3358988: 1
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362788%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [autobid.de] INFO: Scrape type for 3362788: 1
2026-04-01 08:41:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-t5-multivan-2-0-tdi-dsg-highline-3358977/details> (referer: None)
2026-04-01 08:41:19 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 4b8d2d84-0439-53f3-805a-0ec9c5fd31e7 with identifier: 3358977
2026-04-01 08:41:19 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 4b8d2d84-0439-53f3-805a-0ec9c5fd31e7 with scrape type 1
2026-04-01 08:41:19 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 4b8d2d84-0439-53f3-805a-0ec9c5fd31e7
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33590%252F3358977%252F93495757_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33590/3358977/93495757_hd.jpg not downloaded yet
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33590%252F3358977%252F93495767_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33590/3358977/93495767_hd.jpg not downloaded yet
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33590%252F3358977%252F93495785_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33590/3358977/93495785_hd.jpg not downloaded yet
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33590%252F3358977%252F93495783_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33590/3358977/93495783_hd.jpg not downloaded yet
2026-04-01 08:41:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33590%252F3358977%252F93495772_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33590/3358977/93495772_hd.jpg not downloaded yet
2026-04-01 08:41:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 57f1c9d7-eca8-5d13-aef9-9a57a0b96a96 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33599/3359811/93459261_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:41:21 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/vw-e-crafter-35-hochraum-kasten-3362622/details> (referer: None)
2026-04-01 08:41:21 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 08:41:21 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/vw-e-crafter-35-hochraum-kasten-3362622/details> (referer: None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
    current.result = callback(  # type: ignore[misc]
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
    return self._process_spider_exception(response, spider, _failure)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
    result = method(response=response, exception=exception, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
    raise exception
  File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
    result = method(response=response, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
    raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 08:41:23 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 7583a479-884b-5d5f-8f71-44f1d304b5b6 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33452/3345113/93441782_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:41:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 7583a479-884b-5d5f-8f71-44f1d304b5b6 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33452/3345113/93441792_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:41:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=23> (referer: None)
2026-04-01 08:41:24 [autobid.de] INFO: Page 23/143: found 20 items
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358989%27 HTTP/1.1" 200 None
2026-04-01 08:41:24 [autobid.de] INFO: Scrape type for 3358989: 1
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358990%27 HTTP/1.1" 200 None
2026-04-01 08:41:24 [autobid.de] INFO: Scrape type for 3358990: 1
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358991%27 HTTP/1.1" 200 None
2026-04-01 08:41:24 [autobid.de] INFO: Scrape type for 3358991: 1
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358992%27 HTTP/1.1" 200 None
2026-04-01 08:41:24 [autobid.de] INFO: Scrape type for 3358992: 1
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358993%27 HTTP/1.1" 200 None
2026-04-01 08:41:24 [autobid.de] INFO: Scrape type for 3358993: 1
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358994%27 HTTP/1.1" 200 None
2026-04-01 08:41:24 [autobid.de] INFO: Scrape type for 3358994: 1
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358995%27 HTTP/1.1" 200 None
2026-04-01 08:41:24 [autobid.de] INFO: Scrape type for 3358995: 1
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358997%27 HTTP/1.1" 200 None
2026-04-01 08:41:24 [autobid.de] INFO: Scrape type for 3358997: 1
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358998%27 HTTP/1.1" 200 None
2026-04-01 08:41:24 [autobid.de] INFO: Scrape type for 3358998: 1
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358999%27 HTTP/1.1" 200 None
2026-04-01 08:41:24 [autobid.de] INFO: Scrape type for 3358999: 1
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359000%27 HTTP/1.1" 200 None
2026-04-01 08:41:24 [autobid.de] INFO: Scrape type for 3359000: 1
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359001%27 HTTP/1.1" 200 None
2026-04-01 08:41:24 [autobid.de] INFO: Scrape type for 3359001: 1
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359002%27 HTTP/1.1" 200 None
2026-04-01 08:41:24 [autobid.de] INFO: Scrape type for 3359002: 1
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359003%27 HTTP/1.1" 200 None
2026-04-01 08:41:24 [autobid.de] INFO: Scrape type for 3359003: 1
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359004%27 HTTP/1.1" 200 None
2026-04-01 08:41:24 [autobid.de] INFO: Scrape type for 3359004: 1
2026-04-01 08:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359005%27 HTTP/1.1" 200 None
2026-04-01 08:41:25 [autobid.de] INFO: Scrape type for 3359005: 1
2026-04-01 08:41:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359006%27 HTTP/1.1" 200 None
2026-04-01 08:41:25 [autobid.de] INFO: Scrape type for 3359006: 1
2026-04-01 08:41:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359007%27 HTTP/1.1" 200 None
2026-04-01 08:41:25 [autobid.de] INFO: Scrape type for 3359007: 1
2026-04-01 08:41:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359008%27 HTTP/1.1" 200 None
2026-04-01 08:41:25 [autobid.de] INFO: Scrape type for 3359008: 1
2026-04-01 08:41:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359009%27 HTTP/1.1" 200 None
2026-04-01 08:41:25 [autobid.de] INFO: Scrape type for 3359009: 1
2026-04-01 08:41:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 7583a479-884b-5d5f-8f71-44f1d304b5b6 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33452/3345113/93441788_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:41:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 6df12d10-efdc-5862-8caa-04e6bf1ff949 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33452/3345114/93396932_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:41:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/opel-rocks-e-3359009/details> (referer: None)
2026-04-01 08:41:27 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 04b56da1-06bd-5662-afa4-821bd95666c4 with identifier: 3359009
2026-04-01 08:41:27 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 04b56da1-06bd-5662-afa4-821bd95666c4 with scrape type 1
2026-04-01 08:41:27 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 04b56da1-06bd-5662-afa4-821bd95666c4
2026-04-01 08:41:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359009%252F93456728_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359009/93456728_hd.jpg not downloaded yet
2026-04-01 08:41:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359009%252F93456729_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359009/93456729_hd.jpg not downloaded yet
2026-04-01 08:41:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359009%252F93456723_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359009/93456723_hd.jpg not downloaded yet
2026-04-01 08:41:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359009%252F93456724_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359009/93456724_hd.jpg not downloaded yet
2026-04-01 08:41:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359009%252F93456726_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359009/93456726_hd.jpg not downloaded yet
2026-04-01 08:41:28 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=24> (referer: None)
2026-04-01 08:41:28 [autobid.de] INFO: Page 24/143: found 20 items
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359010%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [autobid.de] INFO: Scrape type for 3359010: 1
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359011%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [autobid.de] INFO: Scrape type for 3359011: 1
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359012%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [autobid.de] INFO: Scrape type for 3359012: 1
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359013%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [autobid.de] INFO: Scrape type for 3359013: 1
2026-04-01 08:41:28 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-golf-1-9-tdi-bluemotion-united-3359008/details> (referer: None)
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359014%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [autobid.de] INFO: Scrape type for 3359014: 1
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359015%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [autobid.de] INFO: Scrape type for 3359015: 1
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359016%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [autobid.de] INFO: Scrape type for 3359016: 1
2026-04-01 08:41:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 6df12d10-efdc-5862-8caa-04e6bf1ff949 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33452/3345114/93396929_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359018%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [autobid.de] INFO: Scrape type for 3359018: 1
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359019%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [autobid.de] INFO: Scrape type for 3359019: 1
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359020%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [autobid.de] INFO: Scrape type for 3359020: 1
2026-04-01 08:41:28 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: da1c5f25-bb0f-5c65-b038-f8aa9638d5df with identifier: 3359008
2026-04-01 08:41:28 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item da1c5f25-bb0f-5c65-b038-f8aa9638d5df with scrape type 1
2026-04-01 08:41:28 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item da1c5f25-bb0f-5c65-b038-f8aa9638d5df
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359008%252F93449454_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359008/93449454_hd.jpg not downloaded yet
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359021%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [autobid.de] INFO: Scrape type for 3359021: 1
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359008%252F93449455_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359008/93449455_hd.jpg not downloaded yet
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359022%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [autobid.de] INFO: Scrape type for 3359022: 1
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359008%252F93449444_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359008/93449444_hd.jpg not downloaded yet
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359023%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [autobid.de] INFO: Scrape type for 3359023: 1
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359008%252F93449446_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359008/93449446_hd.jpg not downloaded yet
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359024%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [autobid.de] INFO: Scrape type for 3359024: 1
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359008%252F93449450_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359008/93449450_hd.jpg not downloaded yet
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359025%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [autobid.de] INFO: Scrape type for 3359025: 1
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359026%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [autobid.de] INFO: Scrape type for 3359026: 1
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359027%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [autobid.de] INFO: Scrape type for 3359027: 1
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359028%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [autobid.de] INFO: Scrape type for 3359028: 1
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359029%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [autobid.de] INFO: Scrape type for 3359029: 1
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359030%27 HTTP/1.1" 200 None
2026-04-01 08:41:28 [autobid.de] INFO: Scrape type for 3359030: 1
2026-04-01 08:41:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item ad1ff5f5-7d8d-552d-b22b-4ecdf1d28d85 due to An error occurred while connecting: 104: Connection reset by peer.
2026-04-01 08:41:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ford-fiesta-1-0-trend-3359010/details> (referer: None)
2026-04-01 08:41:29 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 99e8c995-84e4-521b-aa68-8393e737c35e with identifier: 3359010
2026-04-01 08:41:29 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 99e8c995-84e4-521b-aa68-8393e737c35e with scrape type 1
2026-04-01 08:41:29 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 99e8c995-84e4-521b-aa68-8393e737c35e
2026-04-01 08:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359010%252F93451344_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359010/93451344_hd.jpg not downloaded yet
2026-04-01 08:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359010%252F93451340_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359010/93451340_hd.jpg not downloaded yet
2026-04-01 08:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359010%252F93451332_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359010/93451332_hd.jpg not downloaded yet
2026-04-01 08:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359010%252F93451335_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359010/93451335_hd.jpg not downloaded yet
2026-04-01 08:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359010%252F93451337_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359010/93451337_hd.jpg not downloaded yet
2026-04-01 08:41:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-passat-variant-1-5-tsi-dsg-business-3359013/details> (referer: None)
2026-04-01 08:41:31 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 146486de-235e-5b4c-9791-6866a8a76f6d with identifier: 3359013
2026-04-01 08:41:31 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 146486de-235e-5b4c-9791-6866a8a76f6d with scrape type 1
2026-04-01 08:41:31 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 146486de-235e-5b4c-9791-6866a8a76f6d
2026-04-01 08:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359013%252F93450665_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359013/93450665_hd.jpg not downloaded yet
2026-04-01 08:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359013%252F93450670_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359013/93450670_hd.jpg not downloaded yet
2026-04-01 08:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359013%252F93450646_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359013/93450646_hd.jpg not downloaded yet
2026-04-01 08:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359013%252F93450650_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359013/93450650_hd.jpg not downloaded yet
2026-04-01 08:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359013%252F93450658_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359013/93450658_hd.jpg not downloaded yet
2026-04-01 08:41:32 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-golf-variant-gtd-2-0-tdi-dsg-3359025/details> (referer: None)
2026-04-01 08:41:32 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 8a758e0c-e6e2-5db1-8806-d1b543542031 with identifier: 3359025
2026-04-01 08:41:32 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 8a758e0c-e6e2-5db1-8806-d1b543542031 with scrape type 1
2026-04-01 08:41:32 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 8a758e0c-e6e2-5db1-8806-d1b543542031
2026-04-01 08:41:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359025%252F93434965_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359025/93434965_hd.jpg not downloaded yet
2026-04-01 08:41:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359025%252F93434966_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359025/93434966_hd.jpg not downloaded yet
2026-04-01 08:41:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359025%252F93434970_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359025/93434970_hd.jpg not downloaded yet
2026-04-01 08:41:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359025%252F93434968_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359025/93434968_hd.jpg not downloaded yet
2026-04-01 08:41:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359025%252F93434967_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:41:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359025/93434967_hd.jpg not downloaded yet
2026-04-01 08:41:35 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=25> (referer: None)
2026-04-01 08:41:35 [autobid.de] INFO: Page 25/143: found 20 items
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359031%27 HTTP/1.1" 200 None
2026-04-01 08:41:35 [autobid.de] INFO: Scrape type for 3359031: 1
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359032%27 HTTP/1.1" 200 None
2026-04-01 08:41:35 [autobid.de] INFO: Scrape type for 3359032: 1
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359033%27 HTTP/1.1" 200 None
2026-04-01 08:41:35 [autobid.de] INFO: Scrape type for 3359033: 1
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359034%27 HTTP/1.1" 200 None
2026-04-01 08:41:35 [autobid.de] INFO: Scrape type for 3359034: 1
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359035%27 HTTP/1.1" 200 None
2026-04-01 08:41:35 [autobid.de] INFO: Scrape type for 3359035: 1
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359036%27 HTTP/1.1" 200 None
2026-04-01 08:41:35 [autobid.de] INFO: Scrape type for 3359036: 1
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359037%27 HTTP/1.1" 200 None
2026-04-01 08:41:35 [autobid.de] INFO: Scrape type for 3359037: 1
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359038%27 HTTP/1.1" 200 None
2026-04-01 08:41:35 [autobid.de] INFO: Scrape type for 3359038: 1
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359039%27 HTTP/1.1" 200 None
2026-04-01 08:41:35 [autobid.de] INFO: Scrape type for 3359039: 1
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359040%27 HTTP/1.1" 200 None
2026-04-01 08:41:35 [autobid.de] INFO: Scrape type for 3359040: 1
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359041%27 HTTP/1.1" 200 None
2026-04-01 08:41:35 [autobid.de] INFO: Scrape type for 3359041: 1
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359042%27 HTTP/1.1" 200 None
2026-04-01 08:41:35 [autobid.de] INFO: Scrape type for 3359042: 1
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359043%27 HTTP/1.1" 200 None
2026-04-01 08:41:35 [autobid.de] INFO: Scrape type for 3359043: 1
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359046%27 HTTP/1.1" 200 None
2026-04-01 08:41:35 [autobid.de] INFO: Scrape type for 3359046: 1
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359047%27 HTTP/1.1" 200 None
2026-04-01 08:41:35 [autobid.de] INFO: Scrape type for 3359047: 1
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360074%27 HTTP/1.1" 200 None
2026-04-01 08:41:35 [autobid.de] INFO: Scrape type for 3360074: 1
2026-04-01 08:41:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362013%27 HTTP/1.1" 200 None
2026-04-01 08:41:36 [autobid.de] INFO: Scrape type for 3362013: 1
2026-04-01 08:41:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362014%27 HTTP/1.1" 200 None
2026-04-01 08:41:36 [autobid.de] INFO: Scrape type for 3362014: 1
2026-04-01 08:41:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362015%27 HTTP/1.1" 200 None
2026-04-01 08:41:36 [autobid.de] INFO: Scrape type for 3362015: 1
2026-04-01 08:41:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:41:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362016%27 HTTP/1.1" 200 None
2026-04-01 08:41:36 [autobid.de] INFO: Scrape type for 3362016: 1
2026-04-01 08:42:07 [scrapy.extensions.logstats] INFO: Crawled 85 pages (at 12 pages/min), scraped 6 items (at 0 items/min)
2026-04-01 08:42:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 235MiB
2026-04-01 08:43:07 [scrapy.extensions.logstats] INFO: Crawled 85 pages (at 0 pages/min), scraped 6 items (at 0 items/min)
2026-04-01 08:43:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 235MiB
2026-04-01 08:44:07 [scrapy.extensions.logstats] INFO: Crawled 85 pages (at 0 pages/min), scraped 6 items (at 0 items/min)
2026-04-01 08:44:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 235MiB
2026-04-01 08:44:16 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 19c99c22-d350-53e3-ae43-7dc0c6dc4dde due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33553/3355203/93368349_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:44:18 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item ad1ff5f5-7d8d-552d-b22b-4ecdf1d28d85 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33602/3360113/93511479_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:44:18 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-a6-allroad-quattro-45-tdi-s-tronic-7-gang-doppelkupplungsgetriebe-s-tronic-3362016/details> (referer: None)
2026-04-01 08:44:18 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 1b2f9ee8-45c6-5eac-9aeb-2205fc46cf19 with identifier: 3362016
2026-04-01 08:44:18 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 1b2f9ee8-45c6-5eac-9aeb-2205fc46cf19 with scrape type 1
2026-04-01 08:44:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 1b2f9ee8-45c6-5eac-9aeb-2205fc46cf19
2026-04-01 08:44:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33621%252F3362016%252F93461001_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33621/3362016/93461001_hd.jpg not downloaded yet
2026-04-01 08:44:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33621%252F3362016%252F93460990_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33621/3362016/93460990_hd.jpg not downloaded yet
2026-04-01 08:44:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33621%252F3362016%252F93460992_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33621/3362016/93460992_hd.jpg not downloaded yet
2026-04-01 08:44:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33621%252F3362016%252F93461009_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33621/3362016/93461009_hd.jpg not downloaded yet
2026-04-01 08:44:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33621%252F3362016%252F93460993_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33621/3362016/93460993_hd.jpg not downloaded yet
2026-04-01 08:44:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=26> (referer: None)
2026-04-01 08:44:19 [autobid.de] INFO: Page 26/143: found 20 items
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362019%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [autobid.de] INFO: Scrape type for 3362019: 1
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362020%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [autobid.de] INFO: Scrape type for 3362020: 1
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362022%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [autobid.de] INFO: Scrape type for 3362022: 1
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362023%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [autobid.de] INFO: Scrape type for 3362023: 1
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362024%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [autobid.de] INFO: Scrape type for 3362024: 1
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356733%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [autobid.de] INFO: Scrape type for 3356733: 1
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273348104%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [autobid.de] INFO: Scrape type for 3348104: 1
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273351583%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [autobid.de] INFO: Scrape type for 3351583: 1
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357533%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [autobid.de] INFO: Scrape type for 3357533: 1
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359137%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [autobid.de] INFO: Scrape type for 3359137: 1
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359278%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [autobid.de] INFO: Scrape type for 3359278: 1
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273336712%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [autobid.de] INFO: Scrape type for 3336712: 1
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360360%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [autobid.de] INFO: Scrape type for 3360360: 1
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360687%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [autobid.de] INFO: Scrape type for 3360687: 1
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273335105%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [autobid.de] INFO: Scrape type for 3335105: 1
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360688%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [autobid.de] INFO: Scrape type for 3360688: 1
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360689%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [autobid.de] INFO: Scrape type for 3360689: 1
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360690%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [autobid.de] INFO: Scrape type for 3360690: 1
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360691%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [autobid.de] INFO: Scrape type for 3360691: 1
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360693%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [autobid.de] INFO: Scrape type for 3360693: 1
2026-04-01 08:44:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-a6-avant-50-tdi-quattro-tiptronic-sport-automatik-tiptronic-8-stufen-3362015/details> (referer: None)
2026-04-01 08:44:19 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 39c727c8-bd1a-5388-8699-f6c9e1d04f0a with identifier: 3362015
2026-04-01 08:44:19 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 39c727c8-bd1a-5388-8699-f6c9e1d04f0a with scrape type 1
2026-04-01 08:44:19 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 39c727c8-bd1a-5388-8699-f6c9e1d04f0a
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33621%252F3362015%252F93460978_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33621/3362015/93460978_hd.jpg not downloaded yet
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33621%252F3362015%252F93460979_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33621/3362015/93460979_hd.jpg not downloaded yet
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33621%252F3362015%252F93460975_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33621/3362015/93460975_hd.jpg not downloaded yet
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33621%252F3362015%252F93460971_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33621/3362015/93460971_hd.jpg not downloaded yet
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33621%252F3362015%252F93460965_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33621/3362015/93460965_hd.jpg not downloaded yet
2026-04-01 08:44:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 02b5fab7-15ba-5e78-b798-a9e0fc0c92fc due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33604/3360362/93407028_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:44:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=27> (referer: None)
2026-04-01 08:44:23 [autobid.de] INFO: Page 27/143: found 20 items
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360694%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [autobid.de] INFO: Scrape type for 3360694: 1
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360706%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [autobid.de] INFO: Scrape type for 3360706: 1
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360707%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [autobid.de] INFO: Scrape type for 3360707: 1
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361237%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [autobid.de] INFO: Scrape type for 3361237: 1
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361284%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [autobid.de] INFO: Scrape type for 3361284: 1
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361349%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [autobid.de] INFO: Scrape type for 3361349: 1
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361344%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [autobid.de] INFO: Scrape type for 3361344: 1
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361345%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [autobid.de] INFO: Scrape type for 3361345: 1
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361360%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [autobid.de] INFO: Scrape type for 3361360: 1
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361359%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [autobid.de] INFO: Scrape type for 3361359: 1
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361362%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [autobid.de] INFO: Scrape type for 3361362: 1
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362619%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [autobid.de] INFO: Scrape type for 3362619: 1
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362707%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [autobid.de] INFO: Scrape type for 3362707: 1
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363045%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [autobid.de] INFO: Scrape type for 3363045: 1
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363489%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [autobid.de] INFO: Scrape type for 3363489: 1
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273348753%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [autobid.de] INFO: Scrape type for 3348753: 1
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363563%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [autobid.de] INFO: Scrape type for 3363563: 1
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363564%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [autobid.de] INFO: Scrape type for 3363564: 1
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363173%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [autobid.de] INFO: Scrape type for 3363173: 1
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363588%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [autobid.de] INFO: Scrape type for 3363588: 1
2026-04-01 08:44:23 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 02b5fab7-15ba-5e78-b798-a9e0fc0c92fc due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33604/3360362/93407017_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:44:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/dacia-sandero-1-4-ambiance-3363045/details> (referer: None)
2026-04-01 08:44:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: ef3e1fa6-7ef5-553a-a6ab-136c355874a6 with identifier: 3363045
2026-04-01 08:44:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ef3e1fa6-7ef5-553a-a6ab-136c355874a6 with scrape type 1
2026-04-01 08:44:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item ef3e1fa6-7ef5-553a-a6ab-136c355874a6
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33631%252F3363045%252F93491116_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33631/3363045/93491116_hd.jpg not downloaded yet
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33631%252F3363045%252F93491109_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33631/3363045/93491109_hd.jpg not downloaded yet
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33631%252F3363045%252F93491102_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33631/3363045/93491102_hd.jpg not downloaded yet
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33631%252F3363045%252F93491119_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33631/3363045/93491119_hd.jpg not downloaded yet
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33631%252F3363045%252F93491121_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33631/3363045/93491121_hd.jpg not downloaded yet
2026-04-01 08:44:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 02b5fab7-15ba-5e78-b798-a9e0fc0c92fc due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33604/3360362/93407022_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:44:26 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 02b5fab7-15ba-5e78-b798-a9e0fc0c92fc due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33604/3360362/93407026_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:44:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=28> (referer: None)
2026-04-01 08:44:26 [autobid.de] INFO: Page 28/143: found 20 items
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363172%27 HTTP/1.1" 200 None
2026-04-01 08:44:26 [autobid.de] INFO: Scrape type for 3363172: 1
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363727%27 HTTP/1.1" 200 None
2026-04-01 08:44:26 [autobid.de] INFO: Scrape type for 3363727: 1
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364611%27 HTTP/1.1" 200 None
2026-04-01 08:44:26 [autobid.de] INFO: Scrape type for 3364611: 1
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365082%27 HTTP/1.1" 200 None
2026-04-01 08:44:26 [autobid.de] INFO: Scrape type for 3365082: 1
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273329155%27 HTTP/1.1" 200 None
2026-04-01 08:44:26 [autobid.de] INFO: Scrape type for 3329155: 1
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353320%27 HTTP/1.1" 200 None
2026-04-01 08:44:26 [autobid.de] INFO: Scrape type for 3353320: 1
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356980%27 HTTP/1.1" 200 None
2026-04-01 08:44:26 [autobid.de] INFO: Scrape type for 3356980: 1
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273329170%27 HTTP/1.1" 200 None
2026-04-01 08:44:26 [autobid.de] INFO: Scrape type for 3329170: 1
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273350580%27 HTTP/1.1" 200 None
2026-04-01 08:44:26 [autobid.de] INFO: Scrape type for 3350580: 1
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353319%27 HTTP/1.1" 200 None
2026-04-01 08:44:26 [autobid.de] INFO: Scrape type for 3353319: 1
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273315359%27 HTTP/1.1" 200 None
2026-04-01 08:44:26 [autobid.de] INFO: Scrape type for 3315359: 1
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356979%27 HTTP/1.1" 200 None
2026-04-01 08:44:26 [autobid.de] INFO: Scrape type for 3356979: 1
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273344352%27 HTTP/1.1" 200 None
2026-04-01 08:44:26 [autobid.de] INFO: Scrape type for 3344352: 1
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273350582%27 HTTP/1.1" 200 None
2026-04-01 08:44:26 [autobid.de] INFO: Scrape type for 3350582: 1
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358736%27 HTTP/1.1" 200 None
2026-04-01 08:44:26 [autobid.de] INFO: Scrape type for 3358736: 1
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358737%27 HTTP/1.1" 200 None
2026-04-01 08:44:26 [autobid.de] INFO: Scrape type for 3358737: 1
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358740%27 HTTP/1.1" 200 None
2026-04-01 08:44:26 [autobid.de] INFO: Scrape type for 3358740: 1
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361553%27 HTTP/1.1" 200 None
2026-04-01 08:44:26 [autobid.de] INFO: Scrape type for 3361553: 1
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273349611%27 HTTP/1.1" 200 None
2026-04-01 08:44:26 [autobid.de] INFO: Scrape type for 3349611: 1
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361549%27 HTTP/1.1" 200 None
2026-04-01 08:44:26 [autobid.de] INFO: Scrape type for 3361549: 1
2026-04-01 08:44:27 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 02b5fab7-15ba-5e78-b798-a9e0fc0c92fc with identifier: 3360362
2026-04-01 08:44:27 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 08:44:27 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=blue+Pearl+effect HTTP/1.1" 200 None
2026-04-01 08:44:27 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 08:44:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 08:44:27 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 08:44:27 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=Liquefied+Petroleum+Gas+%28LPG%29 HTTP/1.1" 200 None
2026-04-01 08:44:27 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 08:44:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 08:44:27 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 02b5fab7-15ba-5e78-b798-a9e0fc0c92fc with identifier: 3360362
2026-04-01 08:44:28 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: 02b5fab7-15ba-5e78-b798-a9e0fc0c92fc with identifier: 3360362 to the API
2026-04-01 08:44:28 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 08:44:28 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 08:44:28 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33604/3360362/93407023_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 80892',
 'body_type': 'Van / minibus',
 'category': 'Van / minibus',
 'color': 'blue Pearl effect',
 'currency': 'EUR',
 'engine': '2.8L',
 'engine_horse_power': 204,
 'expiration_date': '2026-04-01T11:00:00Z',
 'fuel_type': 'Liquefied Petroleum Gas (LPG)',
 'id': '02b5fab7-15ba-5e78-b798-a9e0fc0c92fc',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'link_to_web_offer': 'https://autobid.de/en/item/vw-t4-multivan-2-8-lpg-automatik-trendline-3360362',
 'listing': 'auction',
 'location': 'Oranienburg',
 'make': 'VW',
 'model': 'T4 Multivan',
 'odometer': 337200,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 5,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33604/3360362/93407023_hd.jpg',
                'https://cdn.autobid.de/data/cars/33604/3360362/93407026_hd.jpg',
                'https://cdn.autobid.de/data/cars/33604/3360362/93407022_hd.jpg',
                'https://cdn.autobid.de/data/cars/33604/3360362/93407017_hd.jpg',
                'https://cdn.autobid.de/data/cars/33604/3360362/93407028_hd.jpg'],
 'photos': [],
 'price': 6500,
 'price_includes_vat': False,
 'registration_date': '2001-04-01',
 'seats_number': 7,
 'seller_name': 'Subsidiary/authorised dealer of the brand(s): VW',
 'title': 'VW T4 Multivan 2,8 LPG Automatik "Trendline"',
 'transmission': 'automatic',
 'vin': None,
 'year': 2001}
2026-04-01 08:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360362%27 HTTP/1.1" 200 None
2026-04-01 08:44:28 [autobid.de] INFO: Saving data for 3360362: {'created_time': 1775033068.992807, 'last_price_update_time': 1775033068.992821, 'auction_closing_time': 1775041200.0}
2026-04-01 08:44:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3360362') HTTP/1.1" 204 0
2026-04-01 08:44:29 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 02b5fab7-15ba-5e78-b798-a9e0fc0c92fc due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33604/3360362/93407023_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:44:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=29> (referer: None)
2026-04-01 08:44:29 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 4b8d2d84-0439-53f3-805a-0ec9c5fd31e7 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33590/3358977/93495772_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:44:29 [autobid.de] INFO: Page 29/143: found 20 items
2026-04-01 08:44:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361552%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [autobid.de] INFO: Scrape type for 3361552: 1
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362631%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [autobid.de] INFO: Scrape type for 3362631: 1
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273349613%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [autobid.de] INFO: Scrape type for 3349613: 1
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361312%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [autobid.de] INFO: Scrape type for 3361312: 1
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345740%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [autobid.de] INFO: Scrape type for 3345740: 1
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356978%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [autobid.de] INFO: Scrape type for 3356978: 1
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363504%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [autobid.de] INFO: Scrape type for 3363504: 1
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361313%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [autobid.de] INFO: Scrape type for 3361313: 1
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358738%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [autobid.de] INFO: Scrape type for 3358738: 1
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358739%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [autobid.de] INFO: Scrape type for 3358739: 1
2026-04-01 08:44:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/volvo-fe-250-automatik-3-kammer-kuhlkoffer-mit-lbw-3361549/details> (referer: None)
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356327%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [autobid.de] INFO: Scrape type for 3356327: 1
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273342569%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [autobid.de] INFO: Scrape type for 3342569: 1
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273342562%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [autobid.de] INFO: Scrape type for 3342562: 1
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363575%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [autobid.de] INFO: Scrape type for 3363575: 1
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273342571%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [autobid.de] INFO: Scrape type for 3342571: 1
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363577%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [autobid.de] INFO: Scrape type for 3363577: 1
2026-04-01 08:44:30 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: a4a6b6ba-f1ae-5dc4-95a8-bad259399e67 with identifier: 3361549
2026-04-01 08:44:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a4a6b6ba-f1ae-5dc4-95a8-bad259399e67 with scrape type 1
2026-04-01 08:44:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item a4a6b6ba-f1ae-5dc4-95a8-bad259399e67
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33616%252F3361549%252F93437010_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33616/3361549/93437010_hd.jpg not downloaded yet
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273342568%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [autobid.de] INFO: Scrape type for 3342568: 1
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33616%252F3361549%252F93437012_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33616/3361549/93437012_hd.jpg not downloaded yet
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273342561%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [autobid.de] INFO: Scrape type for 3342561: 1
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33616%252F3361549%252F93437013_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33616/3361549/93437013_hd.jpg not downloaded yet
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356328%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [autobid.de] INFO: Scrape type for 3356328: 1
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33616%252F3361549%252F93437014_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33616/3361549/93437014_hd.jpg not downloaded yet
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356329%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [autobid.de] INFO: Scrape type for 3356329: 1
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33616%252F3361549%252F93437015_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33616/3361549/93437015_hd.jpg not downloaded yet
2026-04-01 08:44:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 8a758e0c-e6e2-5db1-8806-d1b543542031 due to An error occurred while connecting: 104: Connection reset by peer.
2026-04-01 08:44:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-t5-kasten-kombi-2-0-tdi-3349611/details> (referer: None)
2026-04-01 08:44:31 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: d8b94719-4e80-5480-977d-614cd037e80f with identifier: 3349611
2026-04-01 08:44:31 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item d8b94719-4e80-5480-977d-614cd037e80f with scrape type 1
2026-04-01 08:44:31 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item d8b94719-4e80-5480-977d-614cd037e80f
2026-04-01 08:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33497%252F3349611%252F93311126_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33497/3349611/93311126_hd.jpg not downloaded yet
2026-04-01 08:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33497%252F3349611%252F93311127_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33497/3349611/93311127_hd.jpg not downloaded yet
2026-04-01 08:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33497%252F3349611%252F93311128_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33497/3349611/93311128_hd.jpg not downloaded yet
2026-04-01 08:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33497%252F3349611%252F93311129_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33497/3349611/93311129_hd.jpg not downloaded yet
2026-04-01 08:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33497%252F3349611%252F93311130_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33497/3349611/93311130_hd.jpg not downloaded yet
2026-04-01 08:44:32 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mercedes-benz-vito-kasten-114-cdi-extralang-9g-tronic-pro-3358739/details> (referer: None)
2026-04-01 08:44:32 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: f0d040ac-eba2-5134-a277-ec9b827ac3d9 with identifier: 3358739
2026-04-01 08:44:32 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f0d040ac-eba2-5134-a277-ec9b827ac3d9 with scrape type 1
2026-04-01 08:44:32 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item f0d040ac-eba2-5134-a277-ec9b827ac3d9
2026-04-01 08:44:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33588%252F3358739%252F93435520_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33588/3358739/93435520_hd.jpg not downloaded yet
2026-04-01 08:44:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33588%252F3358739%252F93435521_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33588/3358739/93435521_hd.jpg not downloaded yet
2026-04-01 08:44:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33588%252F3358739%252F93435522_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33588/3358739/93435522_hd.jpg not downloaded yet
2026-04-01 08:44:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33588%252F3358739%252F93435523_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33588/3358739/93435523_hd.jpg not downloaded yet
2026-04-01 08:44:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33588%252F3358739%252F93435524_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33588/3358739/93435524_hd.jpg not downloaded yet
2026-04-01 08:44:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mercedes-benz-sprinter-kasten-l2-fwd-211-cdi-6-gang-fsg-3342561/details> (referer: None)
2026-04-01 08:44:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 12096e12-5a0f-5573-ba9b-a6d61a553d4a with identifier: 3342561
2026-04-01 08:44:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 12096e12-5a0f-5573-ba9b-a6d61a553d4a with scrape type 1
2026-04-01 08:44:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 12096e12-5a0f-5573-ba9b-a6d61a553d4a
2026-04-01 08:44:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33426%252F3342561%252F93193156_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33426/3342561/93193156_hd.jpg not downloaded yet
2026-04-01 08:44:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33426%252F3342561%252F93193157_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33426/3342561/93193157_hd.jpg not downloaded yet
2026-04-01 08:44:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33426%252F3342561%252F93193160_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33426/3342561/93193160_hd.jpg not downloaded yet
2026-04-01 08:44:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33426%252F3342561%252F93193161_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33426/3342561/93193161_hd.jpg not downloaded yet
2026-04-01 08:44:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33426%252F3342561%252F93193244_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:44:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33426/3342561/93193244_hd.jpg not downloaded yet
2026-04-01 08:44:37 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=30> (referer: None)
2026-04-01 08:44:37 [autobid.de] INFO: Page 30/143: found 20 items
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352583%27 HTTP/1.1" 200 None
2026-04-01 08:44:37 [autobid.de] INFO: Scrape type for 3352583: 1
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356330%27 HTTP/1.1" 200 None
2026-04-01 08:44:37 [autobid.de] INFO: Scrape type for 3356330: 1
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273351291%27 HTTP/1.1" 200 None
2026-04-01 08:44:37 [autobid.de] INFO: Scrape type for 3351291: 1
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352582%27 HTTP/1.1" 200 None
2026-04-01 08:44:37 [autobid.de] INFO: Scrape type for 3352582: 1
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273351292%27 HTTP/1.1" 200 None
2026-04-01 08:44:37 [autobid.de] INFO: Scrape type for 3351292: 1
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365123%27 HTTP/1.1" 200 None
2026-04-01 08:44:37 [autobid.de] INFO: Scrape type for 3365123: 1
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273351556%27 HTTP/1.1" 200 None
2026-04-01 08:44:37 [autobid.de] INFO: Scrape type for 3351556: 1
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365560%27 HTTP/1.1" 200 None
2026-04-01 08:44:37 [autobid.de] INFO: Scrape type for 3365560: 1
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273335118%27 HTTP/1.1" 200 None
2026-04-01 08:44:37 [autobid.de] INFO: Scrape type for 3335118: 1
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352581%27 HTTP/1.1" 200 None
2026-04-01 08:44:37 [autobid.de] INFO: Scrape type for 3352581: 1
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359083%27 HTTP/1.1" 200 None
2026-04-01 08:44:37 [autobid.de] INFO: Scrape type for 3359083: 1
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366843%27 HTTP/1.1" 200 None
2026-04-01 08:44:37 [autobid.de] INFO: Scrape type for 3366843: 1
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353626%27 HTTP/1.1" 200 None
2026-04-01 08:44:37 [autobid.de] INFO: Scrape type for 3353626: 1
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355217%27 HTTP/1.1" 200 None
2026-04-01 08:44:37 [autobid.de] INFO: Scrape type for 3355217: 1
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361368%27 HTTP/1.1" 200 None
2026-04-01 08:44:37 [autobid.de] INFO: Scrape type for 3361368: 1
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273349475%27 HTTP/1.1" 200 None
2026-04-01 08:44:37 [autobid.de] INFO: Scrape type for 3349475: 1
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358149%27 HTTP/1.1" 200 None
2026-04-01 08:44:37 [autobid.de] INFO: Scrape type for 3358149: 1
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273351635%27 HTTP/1.1" 200 None
2026-04-01 08:44:37 [autobid.de] INFO: Scrape type for 3351635: 1
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360004%27 HTTP/1.1" 200 None
2026-04-01 08:44:37 [autobid.de] INFO: Scrape type for 3360004: 1
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:44:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361364%27 HTTP/1.1" 200 None
2026-04-01 08:44:37 [autobid.de] INFO: Scrape type for 3361364: 1
2026-04-01 08:45:07 [scrapy.extensions.logstats] INFO: Crawled 97 pages (at 12 pages/min), scraped 7 items (at 1 items/min)
2026-04-01 08:45:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 235MiB
2026-04-01 08:46:07 [scrapy.extensions.logstats] INFO: Crawled 97 pages (at 0 pages/min), scraped 7 items (at 0 items/min)
2026-04-01 08:46:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 235MiB
2026-04-01 08:47:07 [scrapy.extensions.logstats] INFO: Crawled 97 pages (at 0 pages/min), scraped 7 items (at 0 items/min)
2026-04-01 08:47:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 235MiB
2026-04-01 08:47:16 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 4b8d2d84-0439-53f3-805a-0ec9c5fd31e7 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33590/3358977/93495783_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:47:18 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 4b8d2d84-0439-53f3-805a-0ec9c5fd31e7 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33590/3358977/93495785_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:47:18 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/bmw-x5-xdrive40d-sport-automatic-3361364/details> (referer: None)
2026-04-01 08:47:18 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 3044983c-bd34-513c-baf9-7a2e16b469ff with identifier: 3361364
2026-04-01 08:47:18 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 3044983c-bd34-513c-baf9-7a2e16b469ff with scrape type 1
2026-04-01 08:47:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 3044983c-bd34-513c-baf9-7a2e16b469ff
2026-04-01 08:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361364%252F93429342_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361364/93429342_hd.jpg not downloaded yet
2026-04-01 08:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361364%252F93429365_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361364/93429365_hd.jpg not downloaded yet
2026-04-01 08:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361364%252F93429341_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361364/93429341_hd.jpg not downloaded yet
2026-04-01 08:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361364%252F93429358_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361364/93429358_hd.jpg not downloaded yet
2026-04-01 08:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361364%252F93429340_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361364/93429340_hd.jpg not downloaded yet
2026-04-01 08:47:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=31> (referer: None)
2026-04-01 08:47:19 [autobid.de] INFO: Page 31/143: found 20 items
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361346%27 HTTP/1.1" 200 None
2026-04-01 08:47:19 [autobid.de] INFO: Scrape type for 3361346: 1
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360567%27 HTTP/1.1" 200 None
2026-04-01 08:47:19 [autobid.de] INFO: Scrape type for 3360567: 1
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361972%27 HTTP/1.1" 200 None
2026-04-01 08:47:19 [autobid.de] INFO: Scrape type for 3361972: 1
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358152%27 HTTP/1.1" 200 None
2026-04-01 08:47:19 [autobid.de] INFO: Scrape type for 3358152: 1
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358148%27 HTTP/1.1" 200 None
2026-04-01 08:47:19 [autobid.de] INFO: Scrape type for 3358148: 1
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361320%27 HTTP/1.1" 200 None
2026-04-01 08:47:19 [autobid.de] INFO: Scrape type for 3361320: 1
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360568%27 HTTP/1.1" 200 None
2026-04-01 08:47:19 [autobid.de] INFO: Scrape type for 3360568: 1
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359753%27 HTTP/1.1" 200 None
2026-04-01 08:47:19 [autobid.de] INFO: Scrape type for 3359753: 1
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273349457%27 HTTP/1.1" 200 None
2026-04-01 08:47:19 [autobid.de] INFO: Scrape type for 3349457: 1
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358150%27 HTTP/1.1" 200 None
2026-04-01 08:47:19 [autobid.de] INFO: Scrape type for 3358150: 1
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362025%27 HTTP/1.1" 200 None
2026-04-01 08:47:19 [autobid.de] INFO: Scrape type for 3362025: 1
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361427%27 HTTP/1.1" 200 None
2026-04-01 08:47:19 [autobid.de] INFO: Scrape type for 3361427: 1
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354122%27 HTTP/1.1" 200 None
2026-04-01 08:47:19 [autobid.de] INFO: Scrape type for 3354122: 1
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357522%27 HTTP/1.1" 200 None
2026-04-01 08:47:19 [autobid.de] INFO: Scrape type for 3357522: 1
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358153%27 HTTP/1.1" 200 None
2026-04-01 08:47:19 [autobid.de] INFO: Scrape type for 3358153: 1
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362026%27 HTTP/1.1" 200 None
2026-04-01 08:47:19 [autobid.de] INFO: Scrape type for 3362026: 1
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361347%27 HTTP/1.1" 200 None
2026-04-01 08:47:19 [autobid.de] INFO: Scrape type for 3361347: 1
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361321%27 HTTP/1.1" 200 None
2026-04-01 08:47:19 [autobid.de] INFO: Scrape type for 3361321: 1
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358297%27 HTTP/1.1" 200 None
2026-04-01 08:47:19 [autobid.de] INFO: Scrape type for 3358297: 1
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273349704%27 HTTP/1.1" 200 None
2026-04-01 08:47:19 [autobid.de] INFO: Scrape type for 3349704: 1
2026-04-01 08:47:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mini-one-cabriolet-3360004/details> (referer: None)
2026-04-01 08:47:19 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 45a5654b-bce0-506e-bf81-e71bd30baa6c with identifier: 3360004
2026-04-01 08:47:19 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 45a5654b-bce0-506e-bf81-e71bd30baa6c with scrape type 1
2026-04-01 08:47:19 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 45a5654b-bce0-506e-bf81-e71bd30baa6c
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360004%252F93516819_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360004/93516819_hd.jpg not downloaded yet
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360004%252F93516820_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360004/93516820_hd.jpg not downloaded yet
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360004%252F93516821_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360004/93516821_hd.jpg not downloaded yet
2026-04-01 08:47:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360004%252F93516822_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360004/93516822_hd.jpg not downloaded yet
2026-04-01 08:47:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360004%252F93516823_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360004/93516823_hd.jpg not downloaded yet
2026-04-01 08:47:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 4b8d2d84-0439-53f3-805a-0ec9c5fd31e7 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33590/3358977/93495767_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:47:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=32> (referer: None)
2026-04-01 08:47:22 [autobid.de] INFO: Page 32/143: found 20 items
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358335%27 HTTP/1.1" 200 None
2026-04-01 08:47:22 [autobid.de] INFO: Scrape type for 3358335: 1
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355226%27 HTTP/1.1" 200 None
2026-04-01 08:47:22 [autobid.de] INFO: Scrape type for 3355226: 1
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361426%27 HTTP/1.1" 200 None
2026-04-01 08:47:22 [autobid.de] INFO: Scrape type for 3361426: 1
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353345%27 HTTP/1.1" 200 None
2026-04-01 08:47:22 [autobid.de] INFO: Scrape type for 3353345: 1
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273349706%27 HTTP/1.1" 200 None
2026-04-01 08:47:22 [autobid.de] INFO: Scrape type for 3349706: 1
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362802%27 HTTP/1.1" 200 None
2026-04-01 08:47:22 [autobid.de] INFO: Scrape type for 3362802: 1
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361413%27 HTTP/1.1" 200 None
2026-04-01 08:47:22 [autobid.de] INFO: Scrape type for 3361413: 1
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361332%27 HTTP/1.1" 200 None
2026-04-01 08:47:22 [autobid.de] INFO: Scrape type for 3361332: 1
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353654%27 HTTP/1.1" 200 None
2026-04-01 08:47:22 [autobid.de] INFO: Scrape type for 3353654: 1
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358147%27 HTTP/1.1" 200 None
2026-04-01 08:47:22 [autobid.de] INFO: Scrape type for 3358147: 1
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357026%27 HTTP/1.1" 200 None
2026-04-01 08:47:22 [autobid.de] INFO: Scrape type for 3357026: 1
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360572%27 HTTP/1.1" 200 None
2026-04-01 08:47:22 [autobid.de] INFO: Scrape type for 3360572: 1
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359749%27 HTTP/1.1" 200 None
2026-04-01 08:47:22 [autobid.de] INFO: Scrape type for 3359749: 1
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360041%27 HTTP/1.1" 200 None
2026-04-01 08:47:22 [autobid.de] INFO: Scrape type for 3360041: 1
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358241%27 HTTP/1.1" 200 None
2026-04-01 08:47:22 [autobid.de] INFO: Scrape type for 3358241: 1
2026-04-01 08:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358165%27 HTTP/1.1" 200 None
2026-04-01 08:47:23 [autobid.de] INFO: Scrape type for 3358165: 1
2026-04-01 08:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352395%27 HTTP/1.1" 200 None
2026-04-01 08:47:23 [autobid.de] INFO: Scrape type for 3352395: 1
2026-04-01 08:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360005%27 HTTP/1.1" 200 None
2026-04-01 08:47:23 [autobid.de] INFO: Scrape type for 3360005: 1
2026-04-01 08:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361363%27 HTTP/1.1" 200 None
2026-04-01 08:47:23 [autobid.de] INFO: Scrape type for 3361363: 1
2026-04-01 08:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361337%27 HTTP/1.1" 200 None
2026-04-01 08:47:23 [autobid.de] INFO: Scrape type for 3361337: 1
2026-04-01 08:47:23 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 4b8d2d84-0439-53f3-805a-0ec9c5fd31e7 with identifier: 3358977
2026-04-01 08:47:23 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 08:47:23 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=black HTTP/1.1" 200 None
2026-04-01 08:47:23 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 08:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 08:47:23 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 4b8d2d84-0439-53f3-805a-0ec9c5fd31e7 with identifier: 3358977
2026-04-01 08:47:23 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: 4b8d2d84-0439-53f3-805a-0ec9c5fd31e7 with identifier: 3358977 to the API
2026-04-01 08:47:23 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 08:47:24 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 08:47:24 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33590/3358977/93495757_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 81247',
 'body_type': 'Van / minibus',
 'category': 'Van / minibus',
 'color': 'black',
 'currency': 'EUR',
 'emission_standard': 'EURO 5',
 'engine': '2.0L',
 'engine_horse_power': 179,
 'expiration_date': '2026-04-01T12:00:00Z',
 'fuel_type': 'Diesel',
 'id': '4b8d2d84-0439-53f3-805a-0ec9c5fd31e7',
 'is_damaged': False,
 'is_operable': True,
 'is_vat_deductible': False,
 'link_to_web_offer': 'https://autobid.de/en/item/vw-t5-multivan-2-0-tdi-dsg-highline-3358977',
 'listing': 'auction',
 'location': 'Jever',
 'make': 'VW',
 'model': 'T5 Multivan',
 'odometer': 270400,
 'odometer_reading_unit': 'km',
 'origin_country_code': 'DE',
 'original_photo_count': 5,
 'photo_urls': ['https://cdn.autobid.de/data/cars/33590/3358977/93495757_hd.jpg',
                'https://cdn.autobid.de/data/cars/33590/3358977/93495767_hd.jpg',
                'https://cdn.autobid.de/data/cars/33590/3358977/93495785_hd.jpg',
                'https://cdn.autobid.de/data/cars/33590/3358977/93495783_hd.jpg',
                'https://cdn.autobid.de/data/cars/33590/3358977/93495772_hd.jpg'],
 'photos': [],
 'price': 12500,
 'price_includes_vat': False,
 'registration_date': '2012-07-01',
 'seats_number': 7,
 'seller_name': None,
 'title': 'VW T5 Multivan 2,0 TDI DSG  "Highline"',
 'transmission': 'automatic',
 'vin': None,
 'year': 2012}
2026-04-01 08:47:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358977%27 HTTP/1.1" 200 None
2026-04-01 08:47:24 [autobid.de] INFO: Saving data for 3358977: {'created_time': 1775033244.18755, 'last_price_update_time': 1775033244.187567, 'auction_closing_time': 1775044800.0}
2026-04-01 08:47:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3358977') HTTP/1.1" 204 0
2026-04-01 08:47:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 4b8d2d84-0439-53f3-805a-0ec9c5fd31e7 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33590/3358977/93495757_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:47:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 04b56da1-06bd-5662-afa4-821bd95666c4 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33591/3359009/93456726_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:47:25 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-a4-avant-3-0-tdi-quattro-s-tronic-sport-line-3361337/details> (referer: None)
2026-04-01 08:47:25 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 3720a34f-6e0c-576e-8095-0113de7de033 with identifier: 3361337
2026-04-01 08:47:25 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 3720a34f-6e0c-576e-8095-0113de7de033 with scrape type 1
2026-04-01 08:47:25 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 3720a34f-6e0c-576e-8095-0113de7de033
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361337%252F93424634_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361337/93424634_hd.jpg not downloaded yet
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361337%252F93424615_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361337/93424615_hd.jpg not downloaded yet
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361337%252F93424614_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361337/93424614_hd.jpg not downloaded yet
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361337%252F93424613_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361337/93424613_hd.jpg not downloaded yet
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361337%252F93424625_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361337/93424625_hd.jpg not downloaded yet
2026-04-01 08:47:25 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=33> (referer: None)
2026-04-01 08:47:25 [autobid.de] INFO: Page 33/143: found 20 items
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363230%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [autobid.de] INFO: Scrape type for 3363230: 1
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360576%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [autobid.de] INFO: Scrape type for 3360576: 1
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356482%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [autobid.de] INFO: Scrape type for 3356482: 1
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357392%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [autobid.de] INFO: Scrape type for 3357392: 1
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358137%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [autobid.de] INFO: Scrape type for 3358137: 1
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359751%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [autobid.de] INFO: Scrape type for 3359751: 1
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361366%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [autobid.de] INFO: Scrape type for 3361366: 1
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361339%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [autobid.de] INFO: Scrape type for 3361339: 1
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361322%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [autobid.de] INFO: Scrape type for 3361322: 1
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361323%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [autobid.de] INFO: Scrape type for 3361323: 1
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361336%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [autobid.de] INFO: Scrape type for 3361336: 1
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361022%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [autobid.de] INFO: Scrape type for 3361022: 1
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361825%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [autobid.de] INFO: Scrape type for 3361825: 1
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360569%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [autobid.de] INFO: Scrape type for 3360569: 1
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361338%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [autobid.de] INFO: Scrape type for 3361338: 1
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361369%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [autobid.de] INFO: Scrape type for 3361369: 1
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359750%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [autobid.de] INFO: Scrape type for 3359750: 1
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360570%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [autobid.de] INFO: Scrape type for 3360570: 1
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361817%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [autobid.de] INFO: Scrape type for 3361817: 1
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361348%27 HTTP/1.1" 200 None
2026-04-01 08:47:25 [autobid.de] INFO: Scrape type for 3361348: 1
2026-04-01 08:47:26 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 99e8c995-84e4-521b-aa68-8393e737c35e due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33591/3359010/93451337_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:47:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/bmw-x2-xdrivem35i-steptronic-3361363/details> (referer: None)
2026-04-01 08:47:26 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 2389ea9a-e26a-53f0-b807-0010d19e0311 with identifier: 3361363
2026-04-01 08:47:26 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 2389ea9a-e26a-53f0-b807-0010d19e0311 with scrape type 1
2026-04-01 08:47:26 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 2389ea9a-e26a-53f0-b807-0010d19e0311
2026-04-01 08:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361363%252F93429332_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361363/93429332_hd.jpg not downloaded yet
2026-04-01 08:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361363%252F93429320_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361363/93429320_hd.jpg not downloaded yet
2026-04-01 08:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361363%252F93429335_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361363/93429335_hd.jpg not downloaded yet
2026-04-01 08:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361363%252F93429319_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361363/93429319_hd.jpg not downloaded yet
2026-04-01 08:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361363%252F93429311_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361363/93429311_hd.jpg not downloaded yet
2026-04-01 08:47:28 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/bmw-x1-xdrive-18-d-advantage-3361348/details> (referer: None)
2026-04-01 08:47:28 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: f5900340-99c9-536c-98a0-f8a78ac21386 with identifier: 3361348
2026-04-01 08:47:28 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f5900340-99c9-536c-98a0-f8a78ac21386 with scrape type 1
2026-04-01 08:47:28 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item f5900340-99c9-536c-98a0-f8a78ac21386
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361348%252F93426366_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361348/93426366_hd.jpg not downloaded yet
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361348%252F93426373_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361348/93426373_hd.jpg not downloaded yet
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361348%252F93426374_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361348/93426374_hd.jpg not downloaded yet
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361348%252F93426367_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361348/93426367_hd.jpg not downloaded yet
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361348%252F93426378_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361348/93426378_hd.jpg not downloaded yet
2026-04-01 08:47:28 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=34> (referer: None)
2026-04-01 08:47:28 [autobid.de] INFO: Page 34/143: found 20 items
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359748%27 HTTP/1.1" 200 None
2026-04-01 08:47:28 [autobid.de] INFO: Scrape type for 3359748: 1
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360575%27 HTTP/1.1" 200 None
2026-04-01 08:47:28 [autobid.de] INFO: Scrape type for 3360575: 1
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361414%27 HTTP/1.1" 200 None
2026-04-01 08:47:28 [autobid.de] INFO: Scrape type for 3361414: 1
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361333%27 HTTP/1.1" 200 None
2026-04-01 08:47:28 [autobid.de] INFO: Scrape type for 3361333: 1
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361365%27 HTTP/1.1" 200 None
2026-04-01 08:47:28 [autobid.de] INFO: Scrape type for 3361365: 1
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359747%27 HTTP/1.1" 200 None
2026-04-01 08:47:28 [autobid.de] INFO: Scrape type for 3359747: 1
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361367%27 HTTP/1.1" 200 None
2026-04-01 08:47:28 [autobid.de] INFO: Scrape type for 3361367: 1
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361334%27 HTTP/1.1" 200 None
2026-04-01 08:47:28 [autobid.de] INFO: Scrape type for 3361334: 1
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359752%27 HTTP/1.1" 200 None
2026-04-01 08:47:28 [autobid.de] INFO: Scrape type for 3359752: 1
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360574%27 HTTP/1.1" 200 None
2026-04-01 08:47:28 [autobid.de] INFO: Scrape type for 3360574: 1
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360571%27 HTTP/1.1" 200 None
2026-04-01 08:47:28 [autobid.de] INFO: Scrape type for 3360571: 1
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360573%27 HTTP/1.1" 200 None
2026-04-01 08:47:28 [autobid.de] INFO: Scrape type for 3360573: 1
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361675%27 HTTP/1.1" 200 None
2026-04-01 08:47:29 [autobid.de] INFO: Scrape type for 3361675: 1
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365497%27 HTTP/1.1" 200 None
2026-04-01 08:47:29 [autobid.de] INFO: Scrape type for 3365497: 1
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364355%27 HTTP/1.1" 200 None
2026-04-01 08:47:29 [autobid.de] INFO: Scrape type for 3364355: 1
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359999%27 HTTP/1.1" 200 None
2026-04-01 08:47:29 [autobid.de] INFO: Scrape type for 3359999: 1
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364299%27 HTTP/1.1" 200 None
2026-04-01 08:47:29 [autobid.de] INFO: Scrape type for 3364299: 1
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364360%27 HTTP/1.1" 200 None
2026-04-01 08:47:29 [autobid.de] INFO: Scrape type for 3364360: 1
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365496%27 HTTP/1.1" 200 None
2026-04-01 08:47:29 [autobid.de] INFO: Scrape type for 3365496: 1
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362884%27 HTTP/1.1" 200 None
2026-04-01 08:47:29 [autobid.de] INFO: Scrape type for 3362884: 1
2026-04-01 08:47:29 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 146486de-235e-5b4c-9791-6866a8a76f6d due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33591/3359013/93450658_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:47:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-beetle-cabriolet-1-2-tsi-bmt-design-3365496/details> (referer: None)
2026-04-01 08:47:29 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 9a010632-ff96-56e1-9049-922a84565fcf with identifier: 3365496
2026-04-01 08:47:29 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9a010632-ff96-56e1-9049-922a84565fcf with scrape type 1
2026-04-01 08:47:29 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 9a010632-ff96-56e1-9049-922a84565fcf
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33655%252F3365496%252F93620644_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33655/3365496/93620644_hd.jpg not downloaded yet
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33655%252F3365496%252F93620645_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33655/3365496/93620645_hd.jpg not downloaded yet
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33655%252F3365496%252F93620646_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33655/3365496/93620646_hd.jpg not downloaded yet
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33655%252F3365496%252F93620647_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33655/3365496/93620647_hd.jpg not downloaded yet
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33655%252F3365496%252F93620648_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:47:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33655/3365496/93620648_hd.jpg not downloaded yet
2026-04-01 08:47:31 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 1b2f9ee8-45c6-5eac-9aeb-2205fc46cf19 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33621/3362016/93460993_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:47:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=35> (referer: None)
2026-04-01 08:47:33 [autobid.de] INFO: Page 35/143: found 20 items
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357547%27 HTTP/1.1" 200 None
2026-04-01 08:47:33 [autobid.de] INFO: Scrape type for 3357547: 1
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364356%27 HTTP/1.1" 200 None
2026-04-01 08:47:33 [autobid.de] INFO: Scrape type for 3364356: 1
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365362%27 HTTP/1.1" 200 None
2026-04-01 08:47:33 [autobid.de] INFO: Scrape type for 3365362: 1
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365526%27 HTTP/1.1" 200 None
2026-04-01 08:47:33 [autobid.de] INFO: Scrape type for 3365526: 1
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364300%27 HTTP/1.1" 200 None
2026-04-01 08:47:33 [autobid.de] INFO: Scrape type for 3364300: 1
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362803%27 HTTP/1.1" 200 None
2026-04-01 08:47:33 [autobid.de] INFO: Scrape type for 3362803: 1
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361548%27 HTTP/1.1" 200 None
2026-04-01 08:47:33 [autobid.de] INFO: Scrape type for 3361548: 1
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358276%27 HTTP/1.1" 200 None
2026-04-01 08:47:33 [autobid.de] INFO: Scrape type for 3358276: 1
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362656%27 HTTP/1.1" 200 None
2026-04-01 08:47:33 [autobid.de] INFO: Scrape type for 3362656: 1
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273283914%27 HTTP/1.1" 200 None
2026-04-01 08:47:33 [autobid.de] INFO: Scrape type for 3283914: 1
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355814%27 HTTP/1.1" 200 None
2026-04-01 08:47:33 [autobid.de] INFO: Scrape type for 3355814: 1
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358300%27 HTTP/1.1" 200 None
2026-04-01 08:47:33 [autobid.de] INFO: Scrape type for 3358300: 1
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364357%27 HTTP/1.1" 200 None
2026-04-01 08:47:33 [autobid.de] INFO: Scrape type for 3364357: 1
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358287%27 HTTP/1.1" 200 None
2026-04-01 08:47:33 [autobid.de] INFO: Scrape type for 3358287: 1
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358270%27 HTTP/1.1" 200 None
2026-04-01 08:47:33 [autobid.de] INFO: Scrape type for 3358270: 1
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365104%27 HTTP/1.1" 200 None
2026-04-01 08:47:33 [autobid.de] INFO: Scrape type for 3365104: 1
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273351258%27 HTTP/1.1" 200 None
2026-04-01 08:47:33 [autobid.de] INFO: Scrape type for 3351258: 1
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358271%27 HTTP/1.1" 200 None
2026-04-01 08:47:33 [autobid.de] INFO: Scrape type for 3358271: 1
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358278%27 HTTP/1.1" 200 None
2026-04-01 08:47:33 [autobid.de] INFO: Scrape type for 3358278: 1
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:47:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363670%27 HTTP/1.1" 200 None
2026-04-01 08:47:33 [autobid.de] INFO: Scrape type for 3363670: 1
2026-04-01 08:48:07 [scrapy.extensions.logstats] INFO: Crawled 108 pages (at 11 pages/min), scraped 8 items (at 1 items/min)
2026-04-01 08:48:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 238MiB
2026-04-01 08:49:07 [scrapy.extensions.logstats] INFO: Crawled 108 pages (at 0 pages/min), scraped 8 items (at 0 items/min)
2026-04-01 08:49:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 238MiB
2026-04-01 08:50:07 [scrapy.extensions.logstats] INFO: Crawled 108 pages (at 0 pages/min), scraped 8 items (at 0 items/min)
2026-04-01 08:50:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 238MiB
2026-04-01 08:50:16 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 39c727c8-bd1a-5388-8699-f6c9e1d04f0a due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33621/3362015/93460965_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:50:18 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 39c727c8-bd1a-5388-8699-f6c9e1d04f0a due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33621/3362015/93460971_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:50:18 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/peugeot-e-208-active-pack-3363670/details> (referer: None)
2026-04-01 08:50:18 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: cfd24949-e29c-5422-acfe-2a884c475cec with identifier: 3363670
2026-04-01 08:50:18 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item cfd24949-e29c-5422-acfe-2a884c475cec with scrape type 1
2026-04-01 08:50:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item cfd24949-e29c-5422-acfe-2a884c475cec
2026-04-01 08:50:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33637%252F3363670%252F93530219_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33637/3363670/93530219_hd.jpg not downloaded yet
2026-04-01 08:50:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33637%252F3363670%252F93530229_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33637/3363670/93530229_hd.jpg not downloaded yet
2026-04-01 08:50:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33637%252F3363670%252F93530226_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33637/3363670/93530226_hd.jpg not downloaded yet
2026-04-01 08:50:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33637%252F3363670%252F93530228_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33637/3363670/93530228_hd.jpg not downloaded yet
2026-04-01 08:50:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33637%252F3363670%252F93530222_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33637/3363670/93530222_hd.jpg not downloaded yet
2026-04-01 08:50:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=36> (referer: None)
2026-04-01 08:50:19 [autobid.de] INFO: Page 36/143: found 20 items
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364361%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [autobid.de] INFO: Scrape type for 3364361: 1
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347492%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [autobid.de] INFO: Scrape type for 3347492: 1
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362919%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [autobid.de] INFO: Scrape type for 3362919: 1
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347493%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [autobid.de] INFO: Scrape type for 3347493: 1
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361750%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [autobid.de] INFO: Scrape type for 3361750: 1
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347495%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [autobid.de] INFO: Scrape type for 3347495: 1
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362921%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [autobid.de] INFO: Scrape type for 3362921: 1
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347494%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [autobid.de] INFO: Scrape type for 3347494: 1
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361838%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [autobid.de] INFO: Scrape type for 3361838: 1
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347496%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [autobid.de] INFO: Scrape type for 3347496: 1
2026-04-01 08:50:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/volvo-xc40-recharge-pure-electric-2wd-core-3358278/details> (referer: None)
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364253%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [autobid.de] INFO: Scrape type for 3364253: 1
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347497%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [autobid.de] INFO: Scrape type for 3347497: 1
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363265%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [autobid.de] INFO: Scrape type for 3363265: 1
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347498%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [autobid.de] INFO: Scrape type for 3347498: 1
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362898%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [autobid.de] INFO: Scrape type for 3362898: 1
2026-04-01 08:50:19 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 91580a76-054f-57e8-a46f-8b5bdb42dca7 with identifier: 3358278
2026-04-01 08:50:19 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 91580a76-054f-57e8-a46f-8b5bdb42dca7 with scrape type 1
2026-04-01 08:50:19 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 91580a76-054f-57e8-a46f-8b5bdb42dca7
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33583%252F3358278%252F93490651_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33583/3358278/93490651_hd.jpg not downloaded yet
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347456%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [autobid.de] INFO: Scrape type for 3347456: 1
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33583%252F3358278%252F93490652_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33583/3358278/93490652_hd.jpg not downloaded yet
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364364%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [autobid.de] INFO: Scrape type for 3364364: 1
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33583%252F3358278%252F93490653_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33583/3358278/93490653_hd.jpg not downloaded yet
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347499%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [autobid.de] INFO: Scrape type for 3347499: 1
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33583%252F3358278%252F93490654_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33583/3358278/93490654_hd.jpg not downloaded yet
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363292%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [autobid.de] INFO: Scrape type for 3363292: 1
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33583%252F3358278%252F93490655_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33583/3358278/93490655_hd.jpg not downloaded yet
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347501%27 HTTP/1.1" 200 None
2026-04-01 08:50:19 [autobid.de] INFO: Scrape type for 3347501: 1
2026-04-01 08:50:20 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/bmw-2-gran-coupe-218-i-advantage-connected-professional-uvp-51-657-60-3361838/details> (referer: None)
2026-04-01 08:50:20 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 4d089de7-fb99-5dc2-b38a-d4afecbd29dc with identifier: 3361838
2026-04-01 08:50:20 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 4d089de7-fb99-5dc2-b38a-d4afecbd29dc with scrape type 1
2026-04-01 08:50:20 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 3 photos for item 4d089de7-fb99-5dc2-b38a-d4afecbd29dc
2026-04-01 08:50:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33619%252F3361838%252F93456674_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33619/3361838/93456674_hd.jpg not downloaded yet
2026-04-01 08:50:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33619%252F3361838%252F93456675_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33619/3361838/93456675_hd.jpg not downloaded yet
2026-04-01 08:50:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33619%252F3361838%252F93456676_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33619/3361838/93456676_hd.jpg not downloaded yet
2026-04-01 08:50:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item ef3e1fa6-7ef5-553a-a6ab-136c355874a6 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33631/3363045/93491121_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:50:23 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item ef3e1fa6-7ef5-553a-a6ab-136c355874a6 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33631/3363045/93491119_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:50:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=37> (referer: None)
2026-04-01 08:50:23 [autobid.de] INFO: Page 37/143: found 20 items
2026-04-01 08:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364238%27 HTTP/1.1" 200 None
2026-04-01 08:50:23 [autobid.de] INFO: Scrape type for 3364238: 1
2026-04-01 08:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347504%27 HTTP/1.1" 200 None
2026-04-01 08:50:23 [autobid.de] INFO: Scrape type for 3347504: 1
2026-04-01 08:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362820%27 HTTP/1.1" 200 None
2026-04-01 08:50:23 [autobid.de] INFO: Scrape type for 3362820: 1
2026-04-01 08:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347506%27 HTTP/1.1" 200 None
2026-04-01 08:50:23 [autobid.de] INFO: Scrape type for 3347506: 1
2026-04-01 08:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361826%27 HTTP/1.1" 200 None
2026-04-01 08:50:23 [autobid.de] INFO: Scrape type for 3361826: 1
2026-04-01 08:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347507%27 HTTP/1.1" 200 None
2026-04-01 08:50:23 [autobid.de] INFO: Scrape type for 3347507: 1
2026-04-01 08:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365290%27 HTTP/1.1" 200 None
2026-04-01 08:50:23 [autobid.de] INFO: Scrape type for 3365290: 1
2026-04-01 08:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347508%27 HTTP/1.1" 200 None
2026-04-01 08:50:24 [autobid.de] INFO: Scrape type for 3347508: 1
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347563%27 HTTP/1.1" 200 None
2026-04-01 08:50:24 [autobid.de] INFO: Scrape type for 3347563: 1
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347509%27 HTTP/1.1" 200 None
2026-04-01 08:50:24 [autobid.de] INFO: Scrape type for 3347509: 1
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347510%27 HTTP/1.1" 200 None
2026-04-01 08:50:24 [autobid.de] INFO: Scrape type for 3347510: 1
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347511%27 HTTP/1.1" 200 None
2026-04-01 08:50:24 [autobid.de] INFO: Scrape type for 3347511: 1
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347512%27 HTTP/1.1" 200 None
2026-04-01 08:50:24 [autobid.de] INFO: Scrape type for 3347512: 1
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347513%27 HTTP/1.1" 200 None
2026-04-01 08:50:24 [autobid.de] INFO: Scrape type for 3347513: 1
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347514%27 HTTP/1.1" 200 None
2026-04-01 08:50:24 [autobid.de] INFO: Scrape type for 3347514: 1
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347528%27 HTTP/1.1" 200 None
2026-04-01 08:50:24 [autobid.de] INFO: Scrape type for 3347528: 1
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347562%27 HTTP/1.1" 200 None
2026-04-01 08:50:24 [autobid.de] INFO: Scrape type for 3347562: 1
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347564%27 HTTP/1.1" 200 None
2026-04-01 08:50:24 [autobid.de] INFO: Scrape type for 3347564: 1
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357128%27 HTTP/1.1" 200 None
2026-04-01 08:50:24 [autobid.de] INFO: Scrape type for 3357128: 1
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359369%27 HTTP/1.1" 200 None
2026-04-01 08:50:24 [autobid.de] INFO: Scrape type for 3359369: 1
2026-04-01 08:50:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item ef3e1fa6-7ef5-553a-a6ab-136c355874a6 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33631/3363045/93491102_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:50:26 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item d8b94719-4e80-5480-977d-614cd037e80f due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33497/3349611/93311130_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:50:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/rader-reifen-paket-8-satze-3359369/details> (referer: None)
2026-04-01 08:50:26 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 2fedef0a-ce94-5e42-a6ce-c274fd6db6a9 with identifier: 3359369
2026-04-01 08:50:26 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 2fedef0a-ce94-5e42-a6ce-c274fd6db6a9 with scrape type 1
2026-04-01 08:50:26 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 2fedef0a-ce94-5e42-a6ce-c274fd6db6a9
2026-04-01 08:50:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33594%252F3359369%252F93402093_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33594/3359369/93402093_hd.jpg not downloaded yet
2026-04-01 08:50:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=38> (referer: None)
2026-04-01 08:50:26 [autobid.de] INFO: Page 38/143: found 20 items
2026-04-01 08:50:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359370%27 HTTP/1.1" 200 None
2026-04-01 08:50:26 [autobid.de] INFO: Scrape type for 3359370: 1
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361957%27 HTTP/1.1" 200 None
2026-04-01 08:50:27 [autobid.de] INFO: Scrape type for 3361957: 1
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361967%27 HTTP/1.1" 200 None
2026-04-01 08:50:27 [autobid.de] INFO: Scrape type for 3361967: 1
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361958%27 HTTP/1.1" 200 None
2026-04-01 08:50:27 [autobid.de] INFO: Scrape type for 3361958: 1
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361968%27 HTTP/1.1" 200 None
2026-04-01 08:50:27 [autobid.de] INFO: Scrape type for 3361968: 1
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361959%27 HTTP/1.1" 200 None
2026-04-01 08:50:27 [autobid.de] INFO: Scrape type for 3361959: 1
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361966%27 HTTP/1.1" 200 None
2026-04-01 08:50:27 [autobid.de] INFO: Scrape type for 3361966: 1
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361960%27 HTTP/1.1" 200 None
2026-04-01 08:50:27 [autobid.de] INFO: Scrape type for 3361960: 1
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361965%27 HTTP/1.1" 200 None
2026-04-01 08:50:27 [autobid.de] INFO: Scrape type for 3361965: 1
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361961%27 HTTP/1.1" 200 None
2026-04-01 08:50:27 [autobid.de] INFO: Scrape type for 3361961: 1
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361964%27 HTTP/1.1" 200 None
2026-04-01 08:50:27 [autobid.de] INFO: Scrape type for 3361964: 1
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361962%27 HTTP/1.1" 200 None
2026-04-01 08:50:27 [autobid.de] INFO: Scrape type for 3361962: 1
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361963%27 HTTP/1.1" 200 None
2026-04-01 08:50:27 [autobid.de] INFO: Scrape type for 3361963: 1
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362771%27 HTTP/1.1" 200 None
2026-04-01 08:50:27 [autobid.de] INFO: Scrape type for 3362771: 1
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362723%27 HTTP/1.1" 200 None
2026-04-01 08:50:27 [autobid.de] INFO: Scrape type for 3362723: 1
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362775%27 HTTP/1.1" 200 None
2026-04-01 08:50:27 [autobid.de] INFO: Scrape type for 3362775: 1
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362776%27 HTTP/1.1" 200 None
2026-04-01 08:50:27 [autobid.de] INFO: Scrape type for 3362776: 1
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362782%27 HTTP/1.1" 200 None
2026-04-01 08:50:27 [autobid.de] INFO: Scrape type for 3362782: 1
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362773%27 HTTP/1.1" 200 None
2026-04-01 08:50:27 [autobid.de] INFO: Scrape type for 3362773: 1
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362722%27 HTTP/1.1" 200 None
2026-04-01 08:50:27 [autobid.de] INFO: Scrape type for 3362722: 1
2026-04-01 08:50:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/rader-reifen-paket-16-satz-3357128/details> (referer: None)
2026-04-01 08:50:27 [crawlers.middlewares.id_gen_middleware] INFO: Duplicate ID generated for item: 2fedef0a-ce94-5e42-a6ce-c274fd6db6a9 with identifier: 3357128, dropping item
2026-04-01 08:50:29 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item f0d040ac-eba2-5134-a277-ec9b827ac3d9 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33588/3358739/93435524_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:50:29 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-540d-xdrive-touring-sport-automatic-m-sportpaket-3362722/details> (referer: None)
2026-04-01 08:50:29 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 08:50:30 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-540d-xdrive-touring-sport-automatic-m-sportpaket-3362722/details> (referer: None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
    current.result = callback(  # type: ignore[misc]
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
    return self._process_spider_exception(response, spider, _failure)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
    result = method(response=response, exception=exception, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
    raise exception
  File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
    result = method(response=response, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
    raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 08:50:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=39> (referer: None)
2026-04-01 08:50:30 [autobid.de] INFO: Page 39/143: found 20 items
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362721%27 HTTP/1.1" 200 None
2026-04-01 08:50:30 [autobid.de] INFO: Scrape type for 3362721: 1
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362780%27 HTTP/1.1" 200 None
2026-04-01 08:50:30 [autobid.de] INFO: Scrape type for 3362780: 1
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362785%27 HTTP/1.1" 200 None
2026-04-01 08:50:30 [autobid.de] INFO: Scrape type for 3362785: 1
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362772%27 HTTP/1.1" 200 None
2026-04-01 08:50:30 [autobid.de] INFO: Scrape type for 3362772: 1
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362781%27 HTTP/1.1" 200 None
2026-04-01 08:50:30 [autobid.de] INFO: Scrape type for 3362781: 1
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362777%27 HTTP/1.1" 200 None
2026-04-01 08:50:30 [autobid.de] INFO: Scrape type for 3362777: 1
2026-04-01 08:50:30 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/mini-one-pepper-3362773/details> (referer: None)
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362461%27 HTTP/1.1" 200 None
2026-04-01 08:50:30 [autobid.de] INFO: Scrape type for 3362461: 1
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362720%27 HTTP/1.1" 200 None
2026-04-01 08:50:30 [autobid.de] INFO: Scrape type for 3362720: 1
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362460%27 HTTP/1.1" 200 None
2026-04-01 08:50:30 [autobid.de] INFO: Scrape type for 3362460: 1
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362463%27 HTTP/1.1" 200 None
2026-04-01 08:50:30 [autobid.de] INFO: Scrape type for 3362463: 1
2026-04-01 08:50:30 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 08:50:30 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/mini-one-pepper-3362773/details> (referer: None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
    current.result = callback(  # type: ignore[misc]
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
    return self._process_spider_exception(response, spider, _failure)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
    result = method(response=response, exception=exception, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
    raise exception
  File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
    result = method(response=response, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
    raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362784%27 HTTP/1.1" 200 None
2026-04-01 08:50:30 [autobid.de] INFO: Scrape type for 3362784: 1
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362462%27 HTTP/1.1" 200 None
2026-04-01 08:50:30 [autobid.de] INFO: Scrape type for 3362462: 1
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362459%27 HTTP/1.1" 200 None
2026-04-01 08:50:30 [autobid.de] INFO: Scrape type for 3362459: 1
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362787%27 HTTP/1.1" 200 None
2026-04-01 08:50:30 [autobid.de] INFO: Scrape type for 3362787: 1
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362778%27 HTTP/1.1" 200 None
2026-04-01 08:50:30 [autobid.de] INFO: Scrape type for 3362778: 1
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362779%27 HTTP/1.1" 200 None
2026-04-01 08:50:30 [autobid.de] INFO: Scrape type for 3362779: 1
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362774%27 HTTP/1.1" 200 None
2026-04-01 08:50:30 [autobid.de] INFO: Scrape type for 3362774: 1
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362458%27 HTTP/1.1" 200 None
2026-04-01 08:50:30 [autobid.de] INFO: Scrape type for 3362458: 1
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354018%27 HTTP/1.1" 200 None
2026-04-01 08:50:30 [autobid.de] INFO: Scrape type for 3354018: 1
2026-04-01 08:50:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359719%27 HTTP/1.1" 200 None
2026-04-01 08:50:31 [autobid.de] INFO: Scrape type for 3359719: 1
2026-04-01 08:50:31 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 12096e12-5a0f-5573-ba9b-a6d61a553d4a due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33426/3342561/93193244_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:50:31 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-318i-steptronic-3362782/details> (referer: None)
2026-04-01 08:50:31 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 08:50:31 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-318i-steptronic-3362782/details> (referer: None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
    current.result = callback(  # type: ignore[misc]
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
    return self._process_spider_exception(response, spider, _failure)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
    result = method(response=response, exception=exception, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
    raise exception
  File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
    result = method(response=response, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
    raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 08:50:32 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-330i-sport-automatic-m-sport-3362777/details> (referer: None)
2026-04-01 08:50:32 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 08:50:32 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-330i-sport-automatic-m-sport-3362777/details> (referer: None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
    current.result = callback(  # type: ignore[misc]
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
    return self._process_spider_exception(response, spider, _failure)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
    result = method(response=response, exception=exception, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
    raise exception
  File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
    result = method(response=response, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
    raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 08:50:35 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/bmw-116i-3359719/details> (referer: None)
2026-04-01 08:50:35 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: b3377a97-b850-5ce8-a911-19a32361c1ef with identifier: 3359719
2026-04-01 08:50:35 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b3377a97-b850-5ce8-a911-19a32361c1ef with scrape type 1
2026-04-01 08:50:35 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item b3377a97-b850-5ce8-a911-19a32361c1ef
2026-04-01 08:50:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33598%252F3359719%252F93409794_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33598/3359719/93409794_hd.jpg not downloaded yet
2026-04-01 08:50:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33598%252F3359719%252F93409797_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33598/3359719/93409797_hd.jpg not downloaded yet
2026-04-01 08:50:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33598%252F3359719%252F93409788_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33598/3359719/93409788_hd.jpg not downloaded yet
2026-04-01 08:50:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33598%252F3359719%252F93409791_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33598/3359719/93409791_hd.jpg not downloaded yet
2026-04-01 08:50:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33598%252F3359719%252F93409749_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33598/3359719/93409749_hd.jpg not downloaded yet
2026-04-01 08:50:35 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=40> (referer: None)
2026-04-01 08:50:35 [autobid.de] INFO: Page 40/143: found 20 items
2026-04-01 08:50:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358925%27 HTTP/1.1" 200 None
2026-04-01 08:50:35 [autobid.de] INFO: Scrape type for 3358925: 1
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363209%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [autobid.de] INFO: Scrape type for 3363209: 1
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366412%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [autobid.de] INFO: Scrape type for 3366412: 1
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366485%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [autobid.de] INFO: Scrape type for 3366485: 1
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352525%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [autobid.de] INFO: Scrape type for 3352525: 1
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357677%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [autobid.de] INFO: Scrape type for 3357677: 1
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353488%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [autobid.de] INFO: Scrape type for 3353488: 1
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364213%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [autobid.de] INFO: Scrape type for 3364213: 1
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359843%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [autobid.de] INFO: Scrape type for 3359843: 1
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366102%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [autobid.de] INFO: Scrape type for 3366102: 1
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365143%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [autobid.de] INFO: Scrape type for 3365143: 1
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366106%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [autobid.de] INFO: Scrape type for 3366106: 1
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366434%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [autobid.de] INFO: Scrape type for 3366434: 1
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362827%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [autobid.de] INFO: Scrape type for 3362827: 1
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363615%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [autobid.de] INFO: Scrape type for 3363615: 1
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354031%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [autobid.de] INFO: Scrape type for 3354031: 1
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366484%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [autobid.de] INFO: Scrape type for 3366484: 1
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361029%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [autobid.de] INFO: Scrape type for 3361029: 1
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358937%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [autobid.de] INFO: Scrape type for 3358937: 1
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366399%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [autobid.de] INFO: Scrape type for 3366399: 1
2026-04-01 08:50:36 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/seat-mii-1-0-style-3354018/details> (referer: None)
2026-04-01 08:50:36 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: bc9fdfc5-01f0-5ebf-8dbc-c886cf949783 with identifier: 3354018
2026-04-01 08:50:36 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item bc9fdfc5-01f0-5ebf-8dbc-c886cf949783 with scrape type 1
2026-04-01 08:50:36 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item bc9fdfc5-01f0-5ebf-8dbc-c886cf949783
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33541%252F3354018%252F93371259_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33541/3354018/93371259_hd.jpg not downloaded yet
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33541%252F3354018%252F93371263_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33541/3354018/93371263_hd.jpg not downloaded yet
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33541%252F3354018%252F93371272_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33541/3354018/93371272_hd.jpg not downloaded yet
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33541%252F3354018%252F93371268_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33541/3354018/93371268_hd.jpg not downloaded yet
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33541%252F3354018%252F93371265_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:50:36 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33541/3354018/93371265_hd.jpg not downloaded yet
2026-04-01 08:50:37 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-118d-steptronic-sport-line-3362458/details> (referer: None)
2026-04-01 08:50:37 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 08:50:37 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-118d-steptronic-sport-line-3362458/details> (referer: None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
    current.result = callback(  # type: ignore[misc]
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
    return self._process_spider_exception(response, spider, _failure)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
    result = method(response=response, exception=exception, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
    raise exception
  File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
    result = method(response=response, spider=spider)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
    raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 08:50:41 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=41> (referer: None)
2026-04-01 08:50:41 [autobid.de] INFO: Page 41/143: found 20 items
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366425%27 HTTP/1.1" 200 None
2026-04-01 08:50:41 [autobid.de] INFO: Scrape type for 3366425: 1
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366487%27 HTTP/1.1" 200 None
2026-04-01 08:50:41 [autobid.de] INFO: Scrape type for 3366487: 1
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363013%27 HTTP/1.1" 200 None
2026-04-01 08:50:41 [autobid.de] INFO: Scrape type for 3363013: 1
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343417%27 HTTP/1.1" 200 None
2026-04-01 08:50:41 [autobid.de] INFO: Scrape type for 3343417: 1
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364214%27 HTTP/1.1" 200 None
2026-04-01 08:50:41 [autobid.de] INFO: Scrape type for 3364214: 1
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359960%27 HTTP/1.1" 200 None
2026-04-01 08:50:41 [autobid.de] INFO: Scrape type for 3359960: 1
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366103%27 HTTP/1.1" 200 None
2026-04-01 08:50:41 [autobid.de] INFO: Scrape type for 3366103: 1
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365142%27 HTTP/1.1" 200 None
2026-04-01 08:50:41 [autobid.de] INFO: Scrape type for 3365142: 1
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366107%27 HTTP/1.1" 200 None
2026-04-01 08:50:41 [autobid.de] INFO: Scrape type for 3366107: 1
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366435%27 HTTP/1.1" 200 None
2026-04-01 08:50:41 [autobid.de] INFO: Scrape type for 3366435: 1
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366440%27 HTTP/1.1" 200 None
2026-04-01 08:50:41 [autobid.de] INFO: Scrape type for 3366440: 1
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366431%27 HTTP/1.1" 200 None
2026-04-01 08:50:41 [autobid.de] INFO: Scrape type for 3366431: 1
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345589%27 HTTP/1.1" 200 None
2026-04-01 08:50:41 [autobid.de] INFO: Scrape type for 3345589: 1
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273350066%27 HTTP/1.1" 200 None
2026-04-01 08:50:41 [autobid.de] INFO: Scrape type for 3350066: 1
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362662%27 HTTP/1.1" 200 None
2026-04-01 08:50:41 [autobid.de] INFO: Scrape type for 3362662: 1
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359721%27 HTTP/1.1" 200 None
2026-04-01 08:50:41 [autobid.de] INFO: Scrape type for 3359721: 1
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358919%27 HTTP/1.1" 200 None
2026-04-01 08:50:41 [autobid.de] INFO: Scrape type for 3358919: 1
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366405%27 HTTP/1.1" 200 None
2026-04-01 08:50:41 [autobid.de] INFO: Scrape type for 3366405: 1
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366426%27 HTTP/1.1" 200 None
2026-04-01 08:50:41 [autobid.de] INFO: Scrape type for 3366426: 1
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:50:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366488%27 HTTP/1.1" 200 None
2026-04-01 08:50:41 [autobid.de] INFO: Scrape type for 3366488: 1
2026-04-01 08:51:07 [scrapy.extensions.logstats] INFO: Crawled 126 pages (at 18 pages/min), scraped 8 items (at 0 items/min)
2026-04-01 08:51:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 244MiB
2026-04-01 08:52:07 [scrapy.extensions.logstats] INFO: Crawled 126 pages (at 0 pages/min), scraped 8 items (at 0 items/min)
2026-04-01 08:52:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 244MiB
2026-04-01 08:53:07 [scrapy.extensions.logstats] INFO: Crawled 126 pages (at 0 pages/min), scraped 8 items (at 0 items/min)
2026-04-01 08:53:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 244MiB
2026-04-01 08:53:16 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 3044983c-bd34-513c-baf9-7a2e16b469ff due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33614/3361364/93429340_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:53:18 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 45a5654b-bce0-506e-bf81-e71bd30baa6c due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360004/93516823_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:53:18 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/skoda-superb-combi-2-0-tdi-dsg-ambition-3366488/details> (referer: None)
2026-04-01 08:53:18 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: df368f6f-0bd8-5647-ad62-4f1733984bf0 with identifier: 3366488
2026-04-01 08:53:18 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item df368f6f-0bd8-5647-ad62-4f1733984bf0 with scrape type 1
2026-04-01 08:53:18 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item df368f6f-0bd8-5647-ad62-4f1733984bf0
2026-04-01 08:53:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366488%252F93633384_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:53:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366488/93633384_hd.jpg not downloaded yet
2026-04-01 08:53:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366488%252F93633364_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:53:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366488/93633364_hd.jpg not downloaded yet
2026-04-01 08:53:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366488%252F93633376_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:53:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366488/93633376_hd.jpg not downloaded yet
2026-04-01 08:53:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366488%252F93633380_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:53:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366488/93633380_hd.jpg not downloaded yet
2026-04-01 08:53:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366488%252F93633372_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:53:18 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366488/93633372_hd.jpg not downloaded yet
2026-04-01 08:53:18 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING&currentPage=42> (referer: None)
2026-04-01 08:53:19 [autobid.de] INFO: Page 42/143: found 20 items
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361708%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [autobid.de] INFO: Scrape type for 3361708: 1
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359988%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [autobid.de] INFO: Scrape type for 3359988: 1
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363394%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [autobid.de] INFO: Scrape type for 3363394: 1
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365144%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [autobid.de] INFO: Scrape type for 3365144: 1
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366108%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [autobid.de] INFO: Scrape type for 3366108: 1
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366436%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [autobid.de] INFO: Scrape type for 3366436: 1
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358258%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [autobid.de] INFO: Scrape type for 3358258: 1
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345590%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [autobid.de] INFO: Scrape type for 3345590: 1
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352115%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [autobid.de] INFO: Scrape type for 3352115: 1
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366452%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [autobid.de] INFO: Scrape type for 3366452: 1
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361027%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [autobid.de] INFO: Scrape type for 3361027: 1
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358921%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [autobid.de] INFO: Scrape type for 3358921: 1
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366408%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [autobid.de] INFO: Scrape type for 3366408: 1
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366428%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [autobid.de] INFO: Scrape type for 3366428: 1
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366489%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [autobid.de] INFO: Scrape type for 3366489: 1
2026-04-01 08:53:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mercedes-benz-glc-220-d-4matic-9g-tronic-3366426/details> (referer: None)
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363014%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [autobid.de] INFO: Scrape type for 3363014: 1
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366442%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [autobid.de] INFO: Scrape type for 3366442: 1
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273349505%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [autobid.de] INFO: Scrape type for 3349505: 1
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364215%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [autobid.de] INFO: Scrape type for 3364215: 1
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362674%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [autobid.de] INFO: Scrape type for 3362674: 1
2026-04-01 08:53:19 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 46787d95-0d63-5059-b6db-d4d9eb7f08f7 with identifier: 3366426
2026-04-01 08:53:19 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 46787d95-0d63-5059-b6db-d4d9eb7f08f7 with scrape type 1
2026-04-01 08:53:19 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 46787d95-0d63-5059-b6db-d4d9eb7f08f7
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366426%252F93631151_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366426/93631151_hd.jpg not downloaded yet
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366426%252F93631147_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366426/93631147_hd.jpg not downloaded yet
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366426%252F93631141_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366426/93631141_hd.jpg not downloaded yet
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366426%252F93631140_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366426/93631140_hd.jpg not downloaded yet
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:19 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366426%252F93631159_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:53:19 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366426/93631159_hd.jpg not downloaded yet
2026-04-01 08:53:20 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/renault-master-kuhlkoffer-2-3-dci-3366428/details> (referer: None)
2026-04-01 08:53:20 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: e9b6b547-9593-5010-a1be-d55425d7d858 with identifier: 3366428
2026-04-01 08:53:20 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e9b6b547-9593-5010-a1be-d55425d7d858 with scrape type 1
2026-04-01 08:53:20 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item e9b6b547-9593-5010-a1be-d55425d7d858
2026-04-01 08:53:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366428%252F93631218_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:53:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366428/93631218_hd.jpg not downloaded yet
2026-04-01 08:53:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366428%252F93631213_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:53:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366428/93631213_hd.jpg not downloaded yet
2026-04-01 08:53:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 45a5654b-bce0-506e-bf81-e71bd30baa6c due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360004/93516822_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:53:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366428%252F93631220_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:53:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366428/93631220_hd.jpg not downloaded yet
2026-04-01 08:53:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366428%252F93631219_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:53:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366428/93631219_hd.jpg not downloaded yet
2026-04-01 08:53:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 08:53:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366428%252F93631210_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 08:53:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366428/93631210_hd.jpg not downloaded yet
2026-04-01 08:53:23 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 3720a34f-6e0c-576e-8095-0113de7de033 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33614/3361337/93424625_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:53:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 2389ea9a-e26a-53f0-b807-0010d19e0311 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33614/3361363/93429311_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:53:26 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item f5900340-99c9-536c-98a0-f8a78ac21386 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33614/3361348/93426378_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:53:29 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 9a010632-ff96-56e1-9049-922a84565fcf due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33655/3365496/93620648_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:53:31 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 9a010632-ff96-56e1-9049-922a84565fcf due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33655/3365496/93620647_hd.jpg took longer than 180.0 seconds..
2026-04-01 08:54:07 [scrapy.extensions.logstats] INFO: Crawled 130 pages (at 4 pages/min), scraped 8 items (at 0 items/min)
2026-04-01 08:54:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 247MiB
2026-04-01 08:55:07 [scrapy.extensions.logstats] INFO: Crawled 130 pages (at 0 pages/min), scraped 8 items (at 0 items/min)
2026-04-01 08:55:07 [scrapy.extensions.memusage] INFO: Peak memory usage is 247MiB