2026-04-01 06:06:41 [scrapy.utils.log] INFO: Scrapy 2.11.2 started (bot: SourcingV2)
2026-04-01 06:06:41 [scrapy.utils.log] INFO: Versions: lxml 5.2.2.0, libxml2 2.12.6, cssselect 1.2.0, parsel 1.9.1, w3lib 2.1.2, Twisted 24.3.0, Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0], pyOpenSSL 24.1.0 (OpenSSL 3.2.1 30 Jan 2024), cryptography 42.0.7, Platform Linux-5.15.0-1098-azure-x86_64-with-glibc2.36
2026-04-01 06:06:41 [scrapy.addons] INFO: Enabled addons:
[]
2026-04-01 06:06:41 [asyncio] DEBUG: Using selector: EpollSelector
2026-04-01 06:06:41 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.asyncioreactor.AsyncioSelectorReactor
2026-04-01 06:06:41 [scrapy.utils.log] DEBUG: Using asyncio event loop: asyncio.unix_events._UnixSelectorEventLoop
2026-04-01 06:06:41 [scrapy.extensions.telnet] INFO: Telnet Password: 523da0eb4252d1e7
2026-04-01 06:06:42 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
'scrapy.extensions.telnet.TelnetConsole',
'scrapy.extensions.memusage.MemoryUsage',
'scrapy.extensions.feedexport.FeedExporter',
'scrapy.extensions.logstats.LogStats',
'scrapy.extensions.closespider.CloseSpider']
2026-04-01 06:06:42 [scrapy.crawler] INFO: Overridden settings:
{'BOT_NAME': 'SourcingV2',
'CLOSESPIDER_TIMEOUT': 7200,
'DOWNLOAD_DELAY': 1,
'DOWNLOAD_MAXSIZE': 52428800,
'DOWNLOAD_WARNSIZE': 10485760,
'FEED_EXPORT_ENCODING': 'utf-8',
'LOG_FILE': '/var/log/scrapyd/logs/sourcing_v2/autobid.de/2026-04-01T06_06_36.log',
'LOG_FORMATTER': 'crawlers.log_formatter.SourcingLogFormatter',
'MEMUSAGE_LIMIT_MB': 2048,
'MEMUSAGE_WARNING_MB': 1536,
'NEWSPIDER_MODULE': 'spiders',
'REQUEST_FINGERPRINTER_CLASS': 'scrapy_zyte_api.ScrapyZyteAPIRequestFingerprinter',
'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7',
'SPIDER_MODULES': ['spiders', 'auth_check'],
'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor',
'USER_AGENT': ''}
2026-04-01 06:06:42 [scrapy_zyte_api.handler] INFO: Using a Zyte API key starting with '5857011'
2026-04-01 06:06:42 [scrapy_zyte_api.handler] INFO: Using a Zyte API key starting with '5857011'
2026-04-01 06:06:42 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.offsite.OffsiteMiddleware',
'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
'scrapy_zyte_api.ScrapyZyteAPIDownloaderMiddleware',
'scrapy.downloadermiddlewares.retry.RetryMiddleware',
'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
'scrapy.downloadermiddlewares.stats.DownloaderStats']
2026-04-01 06:06:42 [crawlers.middlewares.id_gen_middleware] INFO: Setting up IdGenerationMiddleware
2026-04-01 06:06:42 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
'scrapy_zyte_api.ScrapyZyteAPISpiderMiddleware',
'crawlers.middlewares.monitoring_spider_middleware.MonitoringSpiderMiddleware',
'scrapy.spidermiddlewares.referer.RefererMiddleware',
'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
'scrapy.spidermiddlewares.depth.DepthMiddleware',
'crawlers.middlewares.photo_download_middleware.PhotoDownloadMiddleware',
'crawlers.middlewares.report_download_middleware.ReportDownloadMiddleware',
'crawlers.middlewares.id_gen_middleware.IdGenMiddleware']
2026-04-01 06:06:42 [azure.identity._credentials.environment] INFO: Incomplete environment configuration for EnvironmentCredential. These variables are set: AZURE_TENANT_ID, AZURE_CLIENT_ID
2026-04-01 06:06:42 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2026-04-01 06:06:42 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 2123
2026-04-01 06:06:42 [azure.identity._credentials.chained] INFO: DefaultAzureCredential acquired a token from WorkloadIdentityCredential
2026-04-01 06:06:42 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxsourcingstorageprod.table.core.windows.net:443
2026-04-01 06:06:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:06:42 [crawlers.pipelines.translation_pipeline] INFO: Loading translations for language: auto
2026-04-01 06:06:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /Translations()?$filter=PartitionKey%20eq%20%27auto%27%20and%20RowKey%20eq%20%27auto%27 HTTP/1.1" 200 None
2026-04-01 06:06:42 [crawlers.pipelines.item_rules_pipeline] INFO: Setting up ItemRules Pipeline
2026-04-01 06:06:42 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: invalid_location_for_country.json
2026-04-01 06:06:42 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: damaged_cars_from_auction_title.json
2026-04-01 06:06:42 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: puretech_for_country.json
2026-04-01 06:06:42 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: puretech_for_fr.json
2026-04-01 06:06:42 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: missing_photos.json
2026-04-01 06:06:42 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: damaged_from_info.json
2026-04-01 06:06:42 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: makes_not_allowed.json
2026-04-01 06:06:42 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: not_operable_from_info.json
2026-04-01 06:06:42 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: makes_models_not_allowed.json
2026-04-01 06:06:42 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: keywords_from_title.json
2026-04-01 06:06:42 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: imported_cars.json
2026-04-01 06:06:42 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: invalid_currency.json
2026-04-01 06:06:42 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: missing_mileage.json
2026-04-01 06:06:42 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: keywords_from_auction_title.json
2026-04-01 06:06:42 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: invalid_country_of_origin.json
2026-04-01 06:06:42 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: puretech_for_pt.json
2026-04-01 06:06:42 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: electric_cars.json
2026-04-01 06:06:42 [crawlers.filter_rules.rules_loader] INFO: Loaded rule: missing_color.json
2026-04-01 06:06:42 [crawlers.pipelines.post_to_api] INFO: Setting up PostToApi Pipeline pointing to https://api.app.infinit.cc/api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing
2026-04-01 06:06:42 [scrapy.middleware] INFO: Enabled item pipelines:
['crawlers.pipelines.translation_pipeline.TranslationPipeline',
'crawlers.pipelines.item_rules_pipeline.ItemRulesPipeline',
'crawlers.pipelines.post_to_api.PostToApiPipeline']
2026-04-01 06:06:42 [scrapy.core.engine] INFO: Spider opened
2026-04-01 06:06:42 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 06:06:42 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2026-04-01 06:06:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 143MiB
2026-04-01 06:06:42 [scrapy-playwright] INFO: Starting download handler
2026-04-01 06:06:42 [scrapy-playwright] INFO: Starting download handler
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): publicsuffix.org:443
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://publicsuffix.org:443 "GET /list/public_suffix_list.dat HTTP/1.1" 200 87385
2026-04-01 06:06:50 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=1> (referer: None)
2026-04-01 06:06:50 [autobid.de] INFO: Page 1/145: found 20 items
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361913%27 HTTP/1.1" 200 None
2026-04-01 06:06:50 [autobid.de] INFO: Scrape type for 3361913: 1
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362844%27 HTTP/1.1" 200 None
2026-04-01 06:06:50 [autobid.de] INFO: Scrape type for 3362844: 1
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361914%27 HTTP/1.1" 200 None
2026-04-01 06:06:50 [autobid.de] INFO: Scrape type for 3361914: 1
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362891%27 HTTP/1.1" 200 None
2026-04-01 06:06:50 [autobid.de] INFO: Scrape type for 3362891: 1
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361916%27 HTTP/1.1" 200 None
2026-04-01 06:06:50 [autobid.de] INFO: Scrape type for 3361916: 1
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362889%27 HTTP/1.1" 200 None
2026-04-01 06:06:50 [autobid.de] INFO: Scrape type for 3362889: 1
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361918%27 HTTP/1.1" 200 None
2026-04-01 06:06:50 [autobid.de] INFO: Scrape type for 3361918: 1
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362870%27 HTTP/1.1" 200 None
2026-04-01 06:06:50 [autobid.de] INFO: Scrape type for 3362870: 1
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361921%27 HTTP/1.1" 200 None
2026-04-01 06:06:50 [autobid.de] INFO: Scrape type for 3361921: 1
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362880%27 HTTP/1.1" 200 None
2026-04-01 06:06:50 [autobid.de] INFO: Scrape type for 3362880: 1
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361922%27 HTTP/1.1" 200 None
2026-04-01 06:06:50 [autobid.de] INFO: Scrape type for 3361922: 1
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362887%27 HTTP/1.1" 200 None
2026-04-01 06:06:50 [autobid.de] INFO: Scrape type for 3362887: 1
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361924%27 HTTP/1.1" 200 None
2026-04-01 06:06:50 [autobid.de] INFO: Scrape type for 3361924: 1
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362883%27 HTTP/1.1" 200 None
2026-04-01 06:06:50 [autobid.de] INFO: Scrape type for 3362883: 1
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362838%27 HTTP/1.1" 200 None
2026-04-01 06:06:50 [autobid.de] INFO: Scrape type for 3362838: 1
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362879%27 HTTP/1.1" 200 None
2026-04-01 06:06:50 [autobid.de] INFO: Scrape type for 3362879: 1
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362839%27 HTTP/1.1" 200 None
2026-04-01 06:06:50 [autobid.de] INFO: Scrape type for 3362839: 1
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363263%27 HTTP/1.1" 200 None
2026-04-01 06:06:50 [autobid.de] INFO: Scrape type for 3363263: 1
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362842%27 HTTP/1.1" 200 None
2026-04-01 06:06:50 [autobid.de] INFO: Scrape type for 3362842: 1
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:06:50 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362872%27 HTTP/1.1" 200 None
2026-04-01 06:06:50 [autobid.de] INFO: Scrape type for 3362872: 1
2026-04-01 06:06:50 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-320d-touring-luxury-line-3361913/details> (referer: None)
2026-04-01 06:06:50 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:06:50 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-320d-touring-luxury-line-3361913/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:06:51 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-540d-xdrive-touring-steptronic-business-ivm-m-sportpaket-3362844/details> (referer: None)
2026-04-01 06:06:51 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:06:51 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-540d-xdrive-touring-steptronic-business-ivm-m-sportpaket-3362844/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:06:53 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/mini-cooper-john-cooper-works-trim-3361914/details> (referer: None)
2026-04-01 06:06:53 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:06:53 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/mini-cooper-john-cooper-works-trim-3361914/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:06:54 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-x5-xdrive45e-automatik-3362891/details> (referer: None)
2026-04-01 06:06:54 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:06:54 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-x5-xdrive45e-automatik-3362891/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:06:55 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-218i-gran-coupe-steptronic-m-sport-3361916/details> (referer: None)
2026-04-01 06:06:55 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:06:56 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-218i-gran-coupe-steptronic-m-sport-3361916/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:06:57 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-530d-xdrive-touring-steptronic-luxury-line-3362889/details> (referer: None)
2026-04-01 06:06:57 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:06:57 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-530d-xdrive-touring-steptronic-luxury-line-3362889/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:06:58 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-218i-gran-coupe-steptronic-3361918/details> (referer: None)
2026-04-01 06:06:58 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:06:59 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-218i-gran-coupe-steptronic-3361918/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:00 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-530i-xdrive-touring-automatik-m-sportpaket-3362870/details> (referer: None)
2026-04-01 06:07:00 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:00 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-530i-xdrive-touring-automatik-m-sportpaket-3362870/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:01 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/vw-golf-1-2-tsi-trendline-3361921/details> (referer: None)
2026-04-01 06:07:01 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:01 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/vw-golf-1-2-tsi-trendline-3361921/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:02 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-545e-xdrive-steptronic-business-ivm-m-sportpaket-3362880/details> (referer: None)
2026-04-01 06:07:02 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:03 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-545e-xdrive-steptronic-business-ivm-m-sportpaket-3362880/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:03 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/vw-t-roc-cabriolet-1-0-tsi-move-3361922/details> (referer: None)
2026-04-01 06:07:04 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:04 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/vw-t-roc-cabriolet-1-0-tsi-move-3361922/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:05 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/mercedes-benz-glc-250-4matic-9g-tronic-3362887/details> (referer: None)
2026-04-01 06:07:05 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:05 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/mercedes-benz-glc-250-4matic-9g-tronic-3362887/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:06 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/kia-sportage-1-7-crdi-spirit-3361924/details> (referer: None)
2026-04-01 06:07:06 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:06 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/kia-sportage-1-7-crdi-spirit-3361924/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:07 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/skoda-kodiaq-2-0-tdi-4x4-dsg-sportline-3362883/details> (referer: None)
2026-04-01 06:07:07 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:07 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/skoda-kodiaq-2-0-tdi-4x4-dsg-sportline-3362883/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:08 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-218d-active-tourer-advantage-3362838/details> (referer: None)
2026-04-01 06:07:08 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:09 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-218d-active-tourer-advantage-3362838/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:10 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-318d-touring-steptronic-sport-line-3362879/details> (referer: None)
2026-04-01 06:07:10 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:10 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-318d-touring-steptronic-sport-line-3362879/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:12 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-118i-m-sportpaket-3362872/details> (referer: None)
2026-04-01 06:07:12 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:12 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-118i-m-sportpaket-3362872/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:13 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=2> (referer: None)
2026-04-01 06:07:13 [autobid.de] INFO: Page 2/145: found 20 items
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362893%27 HTTP/1.1" 200 None
2026-04-01 06:07:13 [autobid.de] INFO: Scrape type for 3362893: 1
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362846%27 HTTP/1.1" 200 None
2026-04-01 06:07:13 [autobid.de] INFO: Scrape type for 3362846: 1
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362868%27 HTTP/1.1" 200 None
2026-04-01 06:07:13 [autobid.de] INFO: Scrape type for 3362868: 1
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362848%27 HTTP/1.1" 200 None
2026-04-01 06:07:13 [autobid.de] INFO: Scrape type for 3362848: 1
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362860%27 HTTP/1.1" 200 None
2026-04-01 06:07:13 [autobid.de] INFO: Scrape type for 3362860: 1
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362850%27 HTTP/1.1" 200 None
2026-04-01 06:07:13 [autobid.de] INFO: Scrape type for 3362850: 1
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362877%27 HTTP/1.1" 200 None
2026-04-01 06:07:13 [autobid.de] INFO: Scrape type for 3362877: 1
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362852%27 HTTP/1.1" 200 None
2026-04-01 06:07:13 [autobid.de] INFO: Scrape type for 3362852: 1
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362849%27 HTTP/1.1" 200 None
2026-04-01 06:07:13 [autobid.de] INFO: Scrape type for 3362849: 1
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362854%27 HTTP/1.1" 200 None
2026-04-01 06:07:13 [autobid.de] INFO: Scrape type for 3362854: 1
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362863%27 HTTP/1.1" 200 None
2026-04-01 06:07:13 [autobid.de] INFO: Scrape type for 3362863: 1
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362857%27 HTTP/1.1" 200 None
2026-04-01 06:07:13 [autobid.de] INFO: Scrape type for 3362857: 1
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362906%27 HTTP/1.1" 200 None
2026-04-01 06:07:13 [autobid.de] INFO: Scrape type for 3362906: 1
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362858%27 HTTP/1.1" 200 None
2026-04-01 06:07:13 [autobid.de] INFO: Scrape type for 3362858: 1
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362851%27 HTTP/1.1" 200 None
2026-04-01 06:07:13 [autobid.de] INFO: Scrape type for 3362851: 1
2026-04-01 06:07:13 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/vw-polo-1-0-bmt-3362842/details> (referer: None)
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362859%27 HTTP/1.1" 200 None
2026-04-01 06:07:13 [autobid.de] INFO: Scrape type for 3362859: 1
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362904%27 HTTP/1.1" 200 None
2026-04-01 06:07:13 [autobid.de] INFO: Scrape type for 3362904: 1
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362862%27 HTTP/1.1" 200 None
2026-04-01 06:07:13 [autobid.de] INFO: Scrape type for 3362862: 1
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362853%27 HTTP/1.1" 200 None
2026-04-01 06:07:13 [autobid.de] INFO: Scrape type for 3362853: 1
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:13 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362869%27 HTTP/1.1" 200 None
2026-04-01 06:07:13 [autobid.de] INFO: Scrape type for 3362869: 1
2026-04-01 06:07:13 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:14 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/vw-polo-1-0-bmt-3362842/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:15 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-ix3-inspiring-3363263/details> (referer: None)
2026-04-01 06:07:15 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:15 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-ix3-inspiring-3363263/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:16 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-320d-touring-3362839/details> (referer: None)
2026-04-01 06:07:16 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:16 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-320d-touring-3362839/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:17 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-318d-touring-automatik-3362893/details> (referer: None)
2026-04-01 06:07:17 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:18 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-318d-touring-automatik-3362893/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:18 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/mini-cooper-s-sport-automatic-3362846/details> (referer: None)
2026-04-01 06:07:18 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:18 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/mini-cooper-s-sport-automatic-3362846/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:19 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-218d-active-tourer-automatik-3362868/details> (referer: None)
2026-04-01 06:07:20 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:20 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-218d-active-tourer-automatik-3362868/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:21 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-120d-xdrive-steptronic-m-sport-3362848/details> (referer: None)
2026-04-01 06:07:21 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:21 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-120d-xdrive-steptronic-m-sport-3362848/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:22 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-320d-touring-automatik-sport-line-3362860/details> (referer: None)
2026-04-01 06:07:22 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:22 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-320d-touring-automatik-sport-line-3362860/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:23 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-x2-sdrive20i-steptronic-m-sport-3362850/details> (referer: None)
2026-04-01 06:07:23 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:23 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-x2-sdrive20i-steptronic-m-sport-3362850/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:24 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-x1-xdrive25e-advantage-3362877/details> (referer: None)
2026-04-01 06:07:25 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:25 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-x1-xdrive25e-advantage-3362877/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:26 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-x4-m40d-xdrive-sport-automatic-3362852/details> (referer: None)
2026-04-01 06:07:26 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:26 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-x4-m40d-xdrive-sport-automatic-3362852/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:27 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-218d-active-tourer-automatik-3362849/details> (referer: None)
2026-04-01 06:07:27 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:27 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-218d-active-tourer-automatik-3362849/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:28 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-118d-xdrive-advantage-3362854/details> (referer: None)
2026-04-01 06:07:28 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:28 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-118d-xdrive-advantage-3362854/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:29 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-x1-xdrive20d-automatik-sport-line-3362863/details> (referer: None)
2026-04-01 06:07:29 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:29 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-x1-xdrive20d-automatik-sport-line-3362863/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:30 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-525d-touring-3362857/details> (referer: None)
2026-04-01 06:07:30 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:31 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-525d-touring-3362857/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:31 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/vw-golf-1-0-tsi-bluemotion-technology-automatik-3362906/details> (referer: None)
2026-04-01 06:07:32 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:32 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/vw-golf-1-0-tsi-bluemotion-technology-automatik-3362906/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:33 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-218i-active-tourer-sport-line-3362858/details> (referer: None)
2026-04-01 06:07:33 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:33 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-218i-active-tourer-sport-line-3362858/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:35 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/opel-astra-1-4-3362869/details> (referer: None)
2026-04-01 06:07:35 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:36 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/opel-astra-1-4-3362869/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:36 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=3> (referer: None)
2026-04-01 06:07:36 [autobid.de] INFO: Page 3/145: found 20 items
2026-04-01 06:07:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362907%27 HTTP/1.1" 200 None
2026-04-01 06:07:36 [autobid.de] INFO: Scrape type for 3362907: 1
2026-04-01 06:07:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362871%27 HTTP/1.1" 200 None
2026-04-01 06:07:36 [autobid.de] INFO: Scrape type for 3362871: 1
2026-04-01 06:07:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362861%27 HTTP/1.1" 200 None
2026-04-01 06:07:36 [autobid.de] INFO: Scrape type for 3362861: 1
2026-04-01 06:07:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362874%27 HTTP/1.1" 200 None
2026-04-01 06:07:36 [autobid.de] INFO: Scrape type for 3362874: 1
2026-04-01 06:07:36 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-330d-touring-xdrive-automatik-sport-line-3362853/details> (referer: None)
2026-04-01 06:07:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362881%27 HTTP/1.1" 200 None
2026-04-01 06:07:36 [autobid.de] INFO: Scrape type for 3362881: 1
2026-04-01 06:07:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362876%27 HTTP/1.1" 200 None
2026-04-01 06:07:36 [autobid.de] INFO: Scrape type for 3362876: 1
2026-04-01 06:07:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360125%27 HTTP/1.1" 200 None
2026-04-01 06:07:36 [autobid.de] INFO: Scrape type for 3360125: 1
2026-04-01 06:07:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360127%27 HTTP/1.1" 200 None
2026-04-01 06:07:36 [autobid.de] INFO: Scrape type for 3360127: 1
2026-04-01 06:07:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360130%27 HTTP/1.1" 200 None
2026-04-01 06:07:36 [autobid.de] INFO: Scrape type for 3360130: 1
2026-04-01 06:07:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362250%27 HTTP/1.1" 200 None
2026-04-01 06:07:36 [autobid.de] INFO: Scrape type for 3362250: 1
2026-04-01 06:07:36 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:37 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-330d-touring-xdrive-automatik-sport-line-3362853/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362259%27 HTTP/1.1" 200 None
2026-04-01 06:07:37 [autobid.de] INFO: Scrape type for 3362259: 1
2026-04-01 06:07:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362260%27 HTTP/1.1" 200 None
2026-04-01 06:07:37 [autobid.de] INFO: Scrape type for 3362260: 1
2026-04-01 06:07:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363524%27 HTTP/1.1" 200 None
2026-04-01 06:07:37 [autobid.de] INFO: Scrape type for 3363524: 1
2026-04-01 06:07:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363525%27 HTTP/1.1" 200 None
2026-04-01 06:07:37 [autobid.de] INFO: Scrape type for 3363525: 1
2026-04-01 06:07:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364210%27 HTTP/1.1" 200 None
2026-04-01 06:07:37 [autobid.de] INFO: Scrape type for 3364210: 1
2026-04-01 06:07:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359066%27 HTTP/1.1" 200 None
2026-04-01 06:07:37 [autobid.de] INFO: Scrape type for 3359066: 1
2026-04-01 06:07:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359067%27 HTTP/1.1" 200 None
2026-04-01 06:07:37 [autobid.de] INFO: Scrape type for 3359067: 1
2026-04-01 06:07:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359068%27 HTTP/1.1" 200 None
2026-04-01 06:07:37 [autobid.de] INFO: Scrape type for 3359068: 1
2026-04-01 06:07:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359069%27 HTTP/1.1" 200 None
2026-04-01 06:07:37 [autobid.de] INFO: Scrape type for 3359069: 1
2026-04-01 06:07:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359070%27 HTTP/1.1" 200 None
2026-04-01 06:07:37 [autobid.de] INFO: Scrape type for 3359070: 1
2026-04-01 06:07:37 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/vw-tiguan-allspace-2-0-tdi-4motion-dsg-3362862/details> (referer: None)
2026-04-01 06:07:37 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:38 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/vw-tiguan-allspace-2-0-tdi-4motion-dsg-3362862/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:39 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/opel-mokka-x-1-4-ecotec-3362904/details> (referer: None)
2026-04-01 06:07:39 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:39 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/opel-mokka-x-1-4-ecotec-3362904/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:40 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-330d-touring-steptronic-modern-line-3362859/details> (referer: None)
2026-04-01 06:07:40 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:40 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-330d-touring-steptronic-modern-line-3362859/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:41 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-x3-xdrive20d-steptronic-m-sportpaket-3362851/details> (referer: None)
2026-04-01 06:07:41 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:41 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-x3-xdrive20d-steptronic-m-sportpaket-3362851/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:42 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/fiat-500-1-2-lounge-3362907/details> (referer: None)
2026-04-01 06:07:42 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:42 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/fiat-500-1-2-lounge-3362907/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:42 [scrapy.extensions.logstats] INFO: Crawled 44 pages (at 44 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 06:07:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 173MiB
2026-04-01 06:07:43 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/nissan-qashqai-1-6-dci-automatik-tekna-3362871/details> (referer: None)
2026-04-01 06:07:43 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:43 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/nissan-qashqai-1-6-dci-automatik-tekna-3362871/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:44 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/fiat-500-cabrio-1-2-dualogic-lounge-3362861/details> (referer: None)
2026-04-01 06:07:44 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:44 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/fiat-500-cabrio-1-2-dualogic-lounge-3362861/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:45 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/ford-fiesta-1-0-ecoboost-titanium-3362874/details> (referer: None)
2026-04-01 06:07:45 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:45 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/ford-fiesta-1-0-ecoboost-titanium-3362874/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:46 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-320d-3362881/details> (referer: None)
2026-04-01 06:07:46 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:46 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-320d-3362881/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:48 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-520i-3362876/details> (referer: None)
2026-04-01 06:07:48 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:48 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-520i-3362876/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:49 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-z4-m40i-sport-automatic-3360125/details> (referer: None)
2026-04-01 06:07:49 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:49 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-z4-m40i-sport-automatic-3360125/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:50 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/mini-one-cabrio-3360127/details> (referer: None)
2026-04-01 06:07:50 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:50 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/mini-one-cabrio-3360127/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:52 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-520d-touring-steptronic-3360130/details> (referer: None)
2026-04-01 06:07:52 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:52 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-520d-touring-steptronic-3360130/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:53 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-320d-xdrive-touring-sport-automatic-m-sport-3362250/details> (referer: None)
2026-04-01 06:07:53 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:53 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-320d-xdrive-touring-sport-automatic-m-sport-3362250/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:54 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/opel-astra-1-6-cdti-business-3362259/details> (referer: None)
2026-04-01 06:07:54 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:54 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/opel-astra-1-6-cdti-business-3362259/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:55 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/mini-one-cabrio-steptronic-3362260/details> (referer: None)
2026-04-01 06:07:55 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:55 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/mini-one-cabrio-steptronic-3362260/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:57 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-420d-xdrive-coupe-sport-automatic-m-sport-3359070/details> (referer: None)
2026-04-01 06:07:57 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:58 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-420d-xdrive-coupe-sport-automatic-m-sport-3359070/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:59 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/mercedes-benz-a-220-7g-dct-style-3359069/details> (referer: None)
2026-04-01 06:07:59 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:07:59 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/mercedes-benz-a-220-7g-dct-style-3359069/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:07:59 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=4> (referer: None)
2026-04-01 06:07:59 [autobid.de] INFO: Page 4/145: found 20 items
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360123%27 HTTP/1.1" 200 None
2026-04-01 06:07:59 [autobid.de] INFO: Scrape type for 3360123: 1
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360128%27 HTTP/1.1" 200 None
2026-04-01 06:07:59 [autobid.de] INFO: Scrape type for 3360128: 1
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362251%27 HTTP/1.1" 200 None
2026-04-01 06:07:59 [autobid.de] INFO: Scrape type for 3362251: 1
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362252%27 HTTP/1.1" 200 None
2026-04-01 06:07:59 [autobid.de] INFO: Scrape type for 3362252: 1
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362253%27 HTTP/1.1" 200 None
2026-04-01 06:07:59 [autobid.de] INFO: Scrape type for 3362253: 1
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362254%27 HTTP/1.1" 200 None
2026-04-01 06:07:59 [autobid.de] INFO: Scrape type for 3362254: 1
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362255%27 HTTP/1.1" 200 None
2026-04-01 06:07:59 [autobid.de] INFO: Scrape type for 3362255: 1
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362256%27 HTTP/1.1" 200 None
2026-04-01 06:07:59 [autobid.de] INFO: Scrape type for 3362256: 1
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362257%27 HTTP/1.1" 200 None
2026-04-01 06:07:59 [autobid.de] INFO: Scrape type for 3362257: 1
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362258%27 HTTP/1.1" 200 None
2026-04-01 06:07:59 [autobid.de] INFO: Scrape type for 3362258: 1
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362261%27 HTTP/1.1" 200 None
2026-04-01 06:07:59 [autobid.de] INFO: Scrape type for 3362261: 1
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362262%27 HTTP/1.1" 200 None
2026-04-01 06:07:59 [autobid.de] INFO: Scrape type for 3362262: 1
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362263%27 HTTP/1.1" 200 None
2026-04-01 06:07:59 [autobid.de] INFO: Scrape type for 3362263: 1
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362264%27 HTTP/1.1" 200 None
2026-04-01 06:07:59 [autobid.de] INFO: Scrape type for 3362264: 1
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360154%27 HTTP/1.1" 200 None
2026-04-01 06:07:59 [autobid.de] INFO: Scrape type for 3360154: 1
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362265%27 HTTP/1.1" 200 None
2026-04-01 06:07:59 [autobid.de] INFO: Scrape type for 3362265: 1
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360047%27 HTTP/1.1" 200 None
2026-04-01 06:07:59 [autobid.de] INFO: Scrape type for 3360047: 1
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352727%27 HTTP/1.1" 200 None
2026-04-01 06:07:59 [autobid.de] INFO: Scrape type for 3352727: 1
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360045%27 HTTP/1.1" 200 None
2026-04-01 06:07:59 [autobid.de] INFO: Scrape type for 3360045: 1
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:07:59 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363261%27 HTTP/1.1" 200 None
2026-04-01 06:07:59 [autobid.de] INFO: Scrape type for 3363261: 1
2026-04-01 06:08:00 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/audi-a1-sportback-1-0-tfsi-3359068/details> (referer: None)
2026-04-01 06:08:00 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:08:00 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/audi-a1-sportback-1-0-tfsi-3359068/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:08:01 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-118i-steptronic-advantage-3359067/details> (referer: None)
2026-04-01 06:08:01 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:08:01 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-118i-steptronic-advantage-3359067/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:08:02 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-x3-xdrive20d-steptronic-3359066/details> (referer: None)
2026-04-01 06:08:03 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:08:03 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-x3-xdrive20d-steptronic-3359066/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:08:04 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-x6-xdrive40d-sport-automatic-3364210/details> (referer: None)
2026-04-01 06:08:04 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:08:04 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-x6-xdrive40d-sport-automatic-3364210/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:08:05 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-123d-steptronic-3363525/details> (referer: None)
2026-04-01 06:08:05 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:08:06 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-123d-steptronic-3363525/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:08:06 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-x5-xdrive40i-sport-automatic-m-sportpaket-3363524/details> (referer: None)
2026-04-01 06:08:06 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:08:06 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-x5-xdrive40i-sport-automatic-m-sportpaket-3363524/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:08:07 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-ix-xdrive40-3360123/details> (referer: None)
2026-04-01 06:08:07 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:08:07 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-ix-xdrive40-3360123/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:08:08 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-520d-touring-steptronic-3360128/details> (referer: None)
2026-04-01 06:08:08 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:08:08 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-520d-touring-steptronic-3360128/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:08:09 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-118d-steptronic-advantage-3362251/details> (referer: None)
2026-04-01 06:08:09 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:08:09 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-118d-steptronic-advantage-3362251/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:08:10 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-x3-xdrive20d-steptronic-3362252/details> (referer: None)
2026-04-01 06:08:10 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:08:10 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-x3-xdrive20d-steptronic-3362252/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:08:11 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-320d-xdrive-touring-steptronic-advantage-3362253/details> (referer: None)
2026-04-01 06:08:11 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:08:11 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-320d-xdrive-touring-steptronic-advantage-3362253/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:08:12 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-320d-xdrive-touring-steptronic-sport-line-3362254/details> (referer: None)
2026-04-01 06:08:12 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:08:12 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-320d-xdrive-touring-steptronic-sport-line-3362254/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:08:13 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-530e-xdrive-touring-steptronic-3362255/details> (referer: None)
2026-04-01 06:08:13 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:08:13 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-530e-xdrive-touring-steptronic-3362255/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:08:15 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-ix-xdrive50-3362256/details> (referer: None)
2026-04-01 06:08:15 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:08:15 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-ix-xdrive50-3362256/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:08:16 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-x3-xdrive30e-steptronic-xline-3362257/details> (referer: None)
2026-04-01 06:08:16 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:08:16 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-x3-xdrive30e-steptronic-xline-3362257/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:08:17 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/opel-corsa-1-2-3362258/details> (referer: None)
2026-04-01 06:08:18 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:08:18 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/opel-corsa-1-2-3362258/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:08:20 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/skoda-superb-2-0-tdi-7g-tronic-3363261/details> (referer: None)
2026-04-01 06:08:20 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 5c92f503-4617-5fa1-9021-7c2150620dfb with identifier: 3363261
2026-04-01 06:08:20 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 5c92f503-4617-5fa1-9021-7c2150620dfb with scrape type 1
2026-04-01 06:08:20 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 5c92f503-4617-5fa1-9021-7c2150620dfb
2026-04-01 06:08:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33633%252F3363261%252F93495838_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33633/3363261/93495838_hd.jpg not downloaded yet
2026-04-01 06:08:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33633%252F3363261%252F93495839_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33633/3363261/93495839_hd.jpg not downloaded yet
2026-04-01 06:08:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33633%252F3363261%252F93495840_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33633/3363261/93495840_hd.jpg not downloaded yet
2026-04-01 06:08:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33633%252F3363261%252F93495841_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33633/3363261/93495841_hd.jpg not downloaded yet
2026-04-01 06:08:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33633%252F3363261%252F93495842_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:20 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33633/3363261/93495842_hd.jpg not downloaded yet
2026-04-01 06:08:20 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=5> (referer: None)
2026-04-01 06:08:20 [autobid.de] INFO: Page 5/145: found 20 items
2026-04-01 06:08:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359677%27 HTTP/1.1" 200 None
2026-04-01 06:08:20 [autobid.de] INFO: Scrape type for 3359677: 1
2026-04-01 06:08:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360050%27 HTTP/1.1" 200 None
2026-04-01 06:08:20 [autobid.de] INFO: Scrape type for 3360050: 1
2026-04-01 06:08:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360081%27 HTTP/1.1" 200 None
2026-04-01 06:08:20 [autobid.de] INFO: Scrape type for 3360081: 1
2026-04-01 06:08:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360163%27 HTTP/1.1" 200 None
2026-04-01 06:08:20 [autobid.de] INFO: Scrape type for 3360163: 1
2026-04-01 06:08:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360082%27 HTTP/1.1" 200 None
2026-04-01 06:08:20 [autobid.de] INFO: Scrape type for 3360082: 1
2026-04-01 06:08:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360049%27 HTTP/1.1" 200 None
2026-04-01 06:08:20 [autobid.de] INFO: Scrape type for 3360049: 1
2026-04-01 06:08:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359676%27 HTTP/1.1" 200 None
2026-04-01 06:08:21 [autobid.de] INFO: Scrape type for 3359676: 1
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360164%27 HTTP/1.1" 200 None
2026-04-01 06:08:21 [autobid.de] INFO: Scrape type for 3360164: 1
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355037%27 HTTP/1.1" 200 None
2026-04-01 06:08:21 [autobid.de] INFO: Scrape type for 3355037: 1
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352385%27 HTTP/1.1" 200 None
2026-04-01 06:08:21 [autobid.de] INFO: Scrape type for 3352385: 1
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355043%27 HTTP/1.1" 200 None
2026-04-01 06:08:21 [autobid.de] INFO: Scrape type for 3355043: 1
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360048%27 HTTP/1.1" 200 None
2026-04-01 06:08:21 [autobid.de] INFO: Scrape type for 3360048: 1
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360078%27 HTTP/1.1" 200 None
2026-04-01 06:08:21 [autobid.de] INFO: Scrape type for 3360078: 1
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360046%27 HTTP/1.1" 200 None
2026-04-01 06:08:21 [autobid.de] INFO: Scrape type for 3360046: 1
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359675%27 HTTP/1.1" 200 None
2026-04-01 06:08:21 [autobid.de] INFO: Scrape type for 3359675: 1
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362696%27 HTTP/1.1" 200 None
2026-04-01 06:08:21 [autobid.de] INFO: Scrape type for 3362696: 1
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355044%27 HTTP/1.1" 200 None
2026-04-01 06:08:21 [autobid.de] INFO: Scrape type for 3355044: 1
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362234%27 HTTP/1.1" 200 None
2026-04-01 06:08:21 [autobid.de] INFO: Scrape type for 3362234: 1
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359680%27 HTTP/1.1" 200 None
2026-04-01 06:08:21 [autobid.de] INFO: Scrape type for 3359680: 1
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362700%27 HTTP/1.1" 200 None
2026-04-01 06:08:21 [autobid.de] INFO: Scrape type for 3362700: 1
2026-04-01 06:08:21 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-polo-1-0-fresh-3360045/details> (referer: None)
2026-04-01 06:08:21 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 3125dc43-fb8a-587a-9e88-e45ea136932d with identifier: 3360045
2026-04-01 06:08:21 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 3125dc43-fb8a-587a-9e88-e45ea136932d with scrape type 1
2026-04-01 06:08:21 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 3125dc43-fb8a-587a-9e88-e45ea136932d
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360045%252F93449838_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360045/93449838_hd.jpg not downloaded yet
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360045%252F93449839_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360045/93449839_hd.jpg not downloaded yet
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360045%252F93449840_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360045/93449840_hd.jpg not downloaded yet
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360045%252F93449841_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360045/93449841_hd.jpg not downloaded yet
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360045%252F93449842_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:21 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360045/93449842_hd.jpg not downloaded yet
2026-04-01 06:08:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-tiguan-2-0-tdi-4motion-dsg-r-line-3352727/details> (referer: None)
2026-04-01 06:08:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 76bcaa28-7f4d-5479-87ca-862563a68323 with identifier: 3352727
2026-04-01 06:08:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 76bcaa28-7f4d-5479-87ca-862563a68323 with scrape type 1
2026-04-01 06:08:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 76bcaa28-7f4d-5479-87ca-862563a68323
2026-04-01 06:08:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33528%252F3352727%252F93505310_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33528/3352727/93505310_hd.jpg not downloaded yet
2026-04-01 06:08:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33528%252F3352727%252F93505315_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33528/3352727/93505315_hd.jpg not downloaded yet
2026-04-01 06:08:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33528%252F3352727%252F93505342_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33528/3352727/93505342_hd.jpg not downloaded yet
2026-04-01 06:08:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33528%252F3352727%252F93505328_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33528/3352727/93505328_hd.jpg not downloaded yet
2026-04-01 06:08:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33528%252F3352727%252F93505324_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33528/3352727/93505324_hd.jpg not downloaded yet
2026-04-01 06:08:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/skoda-fabia-1-4-3360047/details> (referer: None)
2026-04-01 06:08:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: e367fd8b-d9b0-5941-bae6-ce20db337086 with identifier: 3360047
2026-04-01 06:08:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e367fd8b-d9b0-5941-bae6-ce20db337086 with scrape type 1
2026-04-01 06:08:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item e367fd8b-d9b0-5941-bae6-ce20db337086
2026-04-01 06:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360047%252F93449623_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360047/93449623_hd.jpg not downloaded yet
2026-04-01 06:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360047%252F93449624_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360047/93449624_hd.jpg not downloaded yet
2026-04-01 06:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360047%252F93449625_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360047/93449625_hd.jpg not downloaded yet
2026-04-01 06:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360047%252F93449626_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360047/93449626_hd.jpg not downloaded yet
2026-04-01 06:08:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360047%252F93449627_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360047/93449627_hd.jpg not downloaded yet
2026-04-01 06:08:25 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-120i-steptronic-3362265/details> (referer: None)
2026-04-01 06:08:25 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:08:25 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-120i-steptronic-3362265/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:08:26 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/kia-stonic-1-0-t-gdi-3360154/details> (referer: None)
2026-04-01 06:08:26 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:08:26 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/kia-stonic-1-0-t-gdi-3360154/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:08:27 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-218i-gran-tourer-steptronic-m-sport-3362264/details> (referer: None)
2026-04-01 06:08:27 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:08:27 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-218i-gran-tourer-steptronic-m-sport-3362264/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:08:28 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-x2-xdrive25e-steptronic-m-sport-x-3362263/details> (referer: None)
2026-04-01 06:08:28 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:08:28 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-x2-xdrive25e-steptronic-m-sport-x-3362263/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:08:29 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/bmw-330d-touring-steptronic-3362262/details> (referer: None)
2026-04-01 06:08:29 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:08:29 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/bmw-330d-touring-steptronic-3362262/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:08:30 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/jeep-compass-1-4-multiair-active-drive-3362261/details> (referer: None)
2026-04-01 06:08:30 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:08:30 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/jeep-compass-1-4-multiair-active-drive-3362261/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:08:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/hyundai-i30-1-4-blue-trend-3359677/details> (referer: None)
2026-04-01 06:08:31 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: e1bc0520-5dc6-5486-9d07-248a98581ca8 with identifier: 3359677
2026-04-01 06:08:31 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e1bc0520-5dc6-5486-9d07-248a98581ca8 with scrape type 1
2026-04-01 06:08:31 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item e1bc0520-5dc6-5486-9d07-248a98581ca8
2026-04-01 06:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33597%252F3359677%252F93637517_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33597/3359677/93637517_hd.jpg not downloaded yet
2026-04-01 06:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33597%252F3359677%252F93637518_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33597/3359677/93637518_hd.jpg not downloaded yet
2026-04-01 06:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33597%252F3359677%252F93637525_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33597/3359677/93637525_hd.jpg not downloaded yet
2026-04-01 06:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33597%252F3359677%252F93637522_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33597/3359677/93637522_hd.jpg not downloaded yet
2026-04-01 06:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33597%252F3359677%252F93637519_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33597/3359677/93637519_hd.jpg not downloaded yet
2026-04-01 06:08:32 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-a4-avant-advanced-40-tdi-s-tronic-3360050/details> (referer: None)
2026-04-01 06:08:32 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 0266a864-f637-53c1-a82a-f69a264e327c with identifier: 3360050
2026-04-01 06:08:32 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 0266a864-f637-53c1-a82a-f69a264e327c with scrape type 1
2026-04-01 06:08:32 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 0266a864-f637-53c1-a82a-f69a264e327c
2026-04-01 06:08:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360050%252F93448775_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360050/93448775_hd.jpg not downloaded yet
2026-04-01 06:08:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360050%252F93448776_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360050/93448776_hd.jpg not downloaded yet
2026-04-01 06:08:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360050%252F93448777_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360050/93448777_hd.jpg not downloaded yet
2026-04-01 06:08:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360050%252F93448778_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360050/93448778_hd.jpg not downloaded yet
2026-04-01 06:08:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360050%252F93448779_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:08:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360050/93448779_hd.jpg not downloaded yet
2026-04-01 06:08:36 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=6> (referer: None)
2026-04-01 06:08:36 [autobid.de] INFO: Page 6/145: found 20 items
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362665%27 HTTP/1.1" 200 None
2026-04-01 06:08:36 [autobid.de] INFO: Scrape type for 3362665: 1
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360080%27 HTTP/1.1" 200 None
2026-04-01 06:08:36 [autobid.de] INFO: Scrape type for 3360080: 1
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362675%27 HTTP/1.1" 200 None
2026-04-01 06:08:36 [autobid.de] INFO: Scrape type for 3362675: 1
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360079%27 HTTP/1.1" 200 None
2026-04-01 06:08:36 [autobid.de] INFO: Scrape type for 3360079: 1
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362685%27 HTTP/1.1" 200 None
2026-04-01 06:08:36 [autobid.de] INFO: Scrape type for 3362685: 1
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359679%27 HTTP/1.1" 200 None
2026-04-01 06:08:36 [autobid.de] INFO: Scrape type for 3359679: 1
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360545%27 HTTP/1.1" 200 None
2026-04-01 06:08:36 [autobid.de] INFO: Scrape type for 3360545: 1
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360084%27 HTTP/1.1" 200 None
2026-04-01 06:08:36 [autobid.de] INFO: Scrape type for 3360084: 1
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362694%27 HTTP/1.1" 200 None
2026-04-01 06:08:36 [autobid.de] INFO: Scrape type for 3362694: 1
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355036%27 HTTP/1.1" 200 None
2026-04-01 06:08:36 [autobid.de] INFO: Scrape type for 3355036: 1
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362668%27 HTTP/1.1" 200 None
2026-04-01 06:08:36 [autobid.de] INFO: Scrape type for 3362668: 1
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360083%27 HTTP/1.1" 200 None
2026-04-01 06:08:36 [autobid.de] INFO: Scrape type for 3360083: 1
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362655%27 HTTP/1.1" 200 None
2026-04-01 06:08:36 [autobid.de] INFO: Scrape type for 3362655: 1
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362653%27 HTTP/1.1" 200 None
2026-04-01 06:08:36 [autobid.de] INFO: Scrape type for 3362653: 1
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360077%27 HTTP/1.1" 200 None
2026-04-01 06:08:36 [autobid.de] INFO: Scrape type for 3360077: 1
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362669%27 HTTP/1.1" 200 None
2026-04-01 06:08:36 [autobid.de] INFO: Scrape type for 3362669: 1
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362691%27 HTTP/1.1" 200 None
2026-04-01 06:08:36 [autobid.de] INFO: Scrape type for 3362691: 1
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362676%27 HTTP/1.1" 200 None
2026-04-01 06:08:36 [autobid.de] INFO: Scrape type for 3362676: 1
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357581%27 HTTP/1.1" 200 None
2026-04-01 06:08:36 [autobid.de] INFO: Scrape type for 3357581: 1
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:08:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352621%27 HTTP/1.1" 200 None
2026-04-01 06:08:36 [autobid.de] INFO: Scrape type for 3352621: 1
2026-04-01 06:08:42 [scrapy.extensions.logstats] INFO: Crawled 88 pages (at 44 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 06:08:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 196MiB
2026-04-01 06:09:42 [scrapy.extensions.logstats] INFO: Crawled 88 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 06:09:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 196MiB
2026-04-01 06:10:42 [scrapy.extensions.logstats] INFO: Crawled 88 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-04-01 06:10:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 196MiB
2026-04-01 06:11:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 5c92f503-4617-5fa1-9021-7c2150620dfb due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33633/3363261/93495838_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:11:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 5c92f503-4617-5fa1-9021-7c2150620dfb due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33633/3363261/93495839_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:11:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-a5-sportback-2-0-tfsi-quattro-3352621/details> (referer: None)
2026-04-01 06:11:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 1b91743c-5ad9-5357-8f18-cf24c4119a5b with identifier: 3352621
2026-04-01 06:11:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 1b91743c-5ad9-5357-8f18-cf24c4119a5b with scrape type 1
2026-04-01 06:11:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 1b91743c-5ad9-5357-8f18-cf24c4119a5b
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33527%252F3352621%252F93438789_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33527/3352621/93438789_hd.jpg not downloaded yet
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33527%252F3352621%252F93438790_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33527/3352621/93438790_hd.jpg not downloaded yet
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33527%252F3352621%252F93438791_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33527/3352621/93438791_hd.jpg not downloaded yet
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33527%252F3352621%252F93438792_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33527/3352621/93438792_hd.jpg not downloaded yet
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33527%252F3352621%252F93438793_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33527/3352621/93438793_hd.jpg not downloaded yet
2026-04-01 06:11:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=7> (referer: None)
2026-04-01 06:11:22 [autobid.de] INFO: Page 7/145: found 20 items
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362699%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [autobid.de] INFO: Scrape type for 3362699: 1
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362663%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [autobid.de] INFO: Scrape type for 3362663: 1
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362680%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [autobid.de] INFO: Scrape type for 3362680: 1
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362684%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [autobid.de] INFO: Scrape type for 3362684: 1
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362931%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [autobid.de] INFO: Scrape type for 3362931: 1
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362678%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [autobid.de] INFO: Scrape type for 3362678: 1
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363856%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [autobid.de] INFO: Scrape type for 3363856: 1
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362689%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [autobid.de] INFO: Scrape type for 3362689: 1
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362654%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [autobid.de] INFO: Scrape type for 3362654: 1
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362688%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [autobid.de] INFO: Scrape type for 3362688: 1
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362657%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [autobid.de] INFO: Scrape type for 3362657: 1
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362667%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [autobid.de] INFO: Scrape type for 3362667: 1
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362701%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [autobid.de] INFO: Scrape type for 3362701: 1
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362666%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [autobid.de] INFO: Scrape type for 3362666: 1
2026-04-01 06:11:22 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 5c92f503-4617-5fa1-9021-7c2150620dfb due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33633/3363261/93495840_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354968%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [autobid.de] INFO: Scrape type for 3354968: 1
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362692%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [autobid.de] INFO: Scrape type for 3362692: 1
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362705%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [autobid.de] INFO: Scrape type for 3362705: 1
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362681%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [autobid.de] INFO: Scrape type for 3362681: 1
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362982%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [autobid.de] INFO: Scrape type for 3362982: 1
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362693%27 HTTP/1.1" 200 None
2026-04-01 06:11:22 [autobid.de] INFO: Scrape type for 3362693: 1
2026-04-01 06:11:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-golf-sportsvan-1-6-tdi-bmt-dsg-comfortline-3357581/details> (referer: None)
2026-04-01 06:11:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 066652a8-7812-57ae-ab95-81add126dea4 with identifier: 3357581
2026-04-01 06:11:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 066652a8-7812-57ae-ab95-81add126dea4 with scrape type 1
2026-04-01 06:11:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 066652a8-7812-57ae-ab95-81add126dea4
2026-04-01 06:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357581%252F93454476_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357581/93454476_hd.jpg not downloaded yet
2026-04-01 06:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357581%252F93454477_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357581/93454477_hd.jpg not downloaded yet
2026-04-01 06:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357581%252F93454478_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357581/93454478_hd.jpg not downloaded yet
2026-04-01 06:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357581%252F93454479_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357581/93454479_hd.jpg not downloaded yet
2026-04-01 06:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357581%252F93454480_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357581/93454480_hd.jpg not downloaded yet
2026-04-01 06:11:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 5c92f503-4617-5fa1-9021-7c2150620dfb due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33633/3363261/93495841_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:11:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-a4-avant-advanced-35-tdi-s-tronic-3362689/details> (referer: None)
2026-04-01 06:11:24 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 0c0d54f5-ec5f-5941-951b-b39e63761ed8 with identifier: 3362689
2026-04-01 06:11:24 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 0c0d54f5-ec5f-5941-951b-b39e63761ed8 with scrape type 1
2026-04-01 06:11:24 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 0c0d54f5-ec5f-5941-951b-b39e63761ed8
2026-04-01 06:11:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362689%252F93483277_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362689/93483277_hd.jpg not downloaded yet
2026-04-01 06:11:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362689%252F93483278_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362689/93483278_hd.jpg not downloaded yet
2026-04-01 06:11:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362689%252F93483279_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362689/93483279_hd.jpg not downloaded yet
2026-04-01 06:11:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362689%252F93483280_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362689/93483280_hd.jpg not downloaded yet
2026-04-01 06:11:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362689%252F93483276_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362689/93483276_hd.jpg not downloaded yet
2026-04-01 06:11:25 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 5c92f503-4617-5fa1-9021-7c2150620dfb with identifier: 3363261
2026-04-01 06:11:25 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 06:11:25 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=blue+Metallic HTTP/1.1" 200 None
2026-04-01 06:11:25 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 06:11:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 06:11:25 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 06:11:25 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=Diesel HTTP/1.1" 200 None
2026-04-01 06:11:25 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 06:11:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 06:11:25 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 5c92f503-4617-5fa1-9021-7c2150620dfb with identifier: 3363261
2026-04-01 06:11:25 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: 5c92f503-4617-5fa1-9021-7c2150620dfb with identifier: 3363261 to the API
2026-04-01 06:11:25 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 06:11:25 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 06:11:25 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33633/3363261/93495842_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 81241',
'body_type': 'Estate',
'category': 'Estate',
'color': 'blue Metallic',
'currency': 'EUR',
'emission_standard': 'EURO 6',
'engine': '2.0L',
'engine_horse_power': 150,
'expiration_date': '2026-04-01T07:30:00Z',
'fuel_type': 'Diesel',
'id': '5c92f503-4617-5fa1-9021-7c2150620dfb',
'is_damaged': False,
'is_operable': True,
'is_vat_included': True,
'link_to_web_offer': 'https://autobid.de/en/item/skoda-superb-2-0-tdi-7g-tronic-3363261',
'listing': 'auction',
'location': 'Rheine',
'make': 'Skoda',
'model': 'Superb',
'odometer': 183500,
'odometer_reading_unit': 'km',
'origin_country_code': 'DE',
'original_photo_count': 5,
'photo_urls': ['https://cdn.autobid.de/data/cars/33633/3363261/93495838_hd.jpg',
'https://cdn.autobid.de/data/cars/33633/3363261/93495839_hd.jpg',
'https://cdn.autobid.de/data/cars/33633/3363261/93495840_hd.jpg',
'https://cdn.autobid.de/data/cars/33633/3363261/93495841_hd.jpg',
'https://cdn.autobid.de/data/cars/33633/3363261/93495842_hd.jpg'],
'photos': [],
'price': 13650,
'price_includes_vat': True,
'registration_date': '2014-02-01',
'seats_number': 5,
'seller_name': 'Ulrich Senger GmbH',
'title': 'Skoda Superb 2,0 TDI 7G-TRONIC',
'transmission': 'automatic',
'vin': None,
'year': 2014}
2026-04-01 06:11:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363261%27 HTTP/1.1" 200 None
2026-04-01 06:11:25 [autobid.de] INFO: Saving data for 3363261: {'created_time': 1775023885.702683, 'last_price_update_time': 1775023885.702696, 'auction_closing_time': 1775028600.0}
2026-04-01 06:11:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3363261') HTTP/1.1" 204 0
2026-04-01 06:11:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 5c92f503-4617-5fa1-9021-7c2150620dfb due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33633/3363261/93495842_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:11:26 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 3125dc43-fb8a-587a-9e88-e45ea136932d due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360045/93449842_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:11:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 76bcaa28-7f4d-5479-87ca-862563a68323 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33528/3352727/93505324_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:11:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=8> (referer: None)
2026-04-01 06:11:28 [autobid.de] INFO: Page 8/145: found 20 items
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362664%27 HTTP/1.1" 200 None
2026-04-01 06:11:28 [autobid.de] INFO: Scrape type for 3362664: 1
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273351648%27 HTTP/1.1" 200 None
2026-04-01 06:11:28 [autobid.de] INFO: Scrape type for 3351648: 1
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358649%27 HTTP/1.1" 200 None
2026-04-01 06:11:28 [autobid.de] INFO: Scrape type for 3358649: 1
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273329146%27 HTTP/1.1" 200 None
2026-04-01 06:11:28 [autobid.de] INFO: Scrape type for 3329146: 1
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362670%27 HTTP/1.1" 200 None
2026-04-01 06:11:28 [autobid.de] INFO: Scrape type for 3362670: 1
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362686%27 HTTP/1.1" 200 None
2026-04-01 06:11:28 [autobid.de] INFO: Scrape type for 3362686: 1
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362697%27 HTTP/1.1" 200 None
2026-04-01 06:11:28 [autobid.de] INFO: Scrape type for 3362697: 1
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362677%27 HTTP/1.1" 200 None
2026-04-01 06:11:28 [autobid.de] INFO: Scrape type for 3362677: 1
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362690%27 HTTP/1.1" 200 None
2026-04-01 06:11:28 [autobid.de] INFO: Scrape type for 3362690: 1
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362687%27 HTTP/1.1" 200 None
2026-04-01 06:11:28 [autobid.de] INFO: Scrape type for 3362687: 1
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362671%27 HTTP/1.1" 200 None
2026-04-01 06:11:28 [autobid.de] INFO: Scrape type for 3362671: 1
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352620%27 HTTP/1.1" 200 None
2026-04-01 06:11:28 [autobid.de] INFO: Scrape type for 3352620: 1
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362673%27 HTTP/1.1" 200 None
2026-04-01 06:11:28 [autobid.de] INFO: Scrape type for 3362673: 1
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358648%27 HTTP/1.1" 200 None
2026-04-01 06:11:28 [autobid.de] INFO: Scrape type for 3358648: 1
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362672%27 HTTP/1.1" 200 None
2026-04-01 06:11:28 [autobid.de] INFO: Scrape type for 3362672: 1
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363627%27 HTTP/1.1" 200 None
2026-04-01 06:11:28 [autobid.de] INFO: Scrape type for 3363627: 1
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354970%27 HTTP/1.1" 200 None
2026-04-01 06:11:28 [autobid.de] INFO: Scrape type for 3354970: 1
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364508%27 HTTP/1.1" 200 None
2026-04-01 06:11:28 [autobid.de] INFO: Scrape type for 3364508: 1
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364507%27 HTTP/1.1" 200 None
2026-04-01 06:11:28 [autobid.de] INFO: Scrape type for 3364507: 1
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364506%27 HTTP/1.1" 200 None
2026-04-01 06:11:28 [autobid.de] INFO: Scrape type for 3364506: 1
2026-04-01 06:11:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item e367fd8b-d9b0-5941-bae6-ce20db337086 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360047/93449627_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:11:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=9> (referer: None)
2026-04-01 06:11:31 [autobid.de] INFO: Page 9/145: found 20 items
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364509%27 HTTP/1.1" 200 None
2026-04-01 06:11:31 [autobid.de] INFO: Scrape type for 3364509: 1
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364510%27 HTTP/1.1" 200 None
2026-04-01 06:11:31 [autobid.de] INFO: Scrape type for 3364510: 1
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365440%27 HTTP/1.1" 200 None
2026-04-01 06:11:31 [autobid.de] INFO: Scrape type for 3365440: 1
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366099%27 HTTP/1.1" 200 None
2026-04-01 06:11:31 [autobid.de] INFO: Scrape type for 3366099: 1
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366092%27 HTTP/1.1" 200 None
2026-04-01 06:11:31 [autobid.de] INFO: Scrape type for 3366092: 1
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366094%27 HTTP/1.1" 200 None
2026-04-01 06:11:31 [autobid.de] INFO: Scrape type for 3366094: 1
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366091%27 HTTP/1.1" 200 None
2026-04-01 06:11:31 [autobid.de] INFO: Scrape type for 3366091: 1
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366093%27 HTTP/1.1" 200 None
2026-04-01 06:11:31 [autobid.de] INFO: Scrape type for 3366093: 1
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365151%27 HTTP/1.1" 200 None
2026-04-01 06:11:31 [autobid.de] INFO: Scrape type for 3365151: 1
2026-04-01 06:11:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-passat-variant-2-0-tdi-bluemotion-comfortline-3364506/details> (referer: None)
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357557%27 HTTP/1.1" 200 None
2026-04-01 06:11:31 [autobid.de] INFO: Scrape type for 3357557: 1
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355812%27 HTTP/1.1" 200 None
2026-04-01 06:11:31 [autobid.de] INFO: Scrape type for 3355812: 1
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355807%27 HTTP/1.1" 200 None
2026-04-01 06:11:31 [autobid.de] INFO: Scrape type for 3355807: 1
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366329%27 HTTP/1.1" 200 None
2026-04-01 06:11:31 [autobid.de] INFO: Scrape type for 3366329: 1
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358779%27 HTTP/1.1" 200 None
2026-04-01 06:11:31 [autobid.de] INFO: Scrape type for 3358779: 1
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364610%27 HTTP/1.1" 200 None
2026-04-01 06:11:31 [autobid.de] INFO: Scrape type for 3364610: 1
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358915%27 HTTP/1.1" 200 None
2026-04-01 06:11:31 [autobid.de] INFO: Scrape type for 3358915: 1
2026-04-01 06:11:31 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 22e82ada-18d6-5a23-ab58-5f27f8a04788 with identifier: 3364506
2026-04-01 06:11:31 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 22e82ada-18d6-5a23-ab58-5f27f8a04788 with scrape type 1
2026-04-01 06:11:31 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 22e82ada-18d6-5a23-ab58-5f27f8a04788
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364506%252F93584963_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364506/93584963_hd.jpg not downloaded yet
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364753%27 HTTP/1.1" 200 None
2026-04-01 06:11:31 [autobid.de] INFO: Scrape type for 3364753: 1
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364506%252F93584964_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364506/93584964_hd.jpg not downloaded yet
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366334%27 HTTP/1.1" 200 None
2026-04-01 06:11:31 [autobid.de] INFO: Scrape type for 3366334: 1
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364506%252F93584965_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364506/93584965_hd.jpg not downloaded yet
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358778%27 HTTP/1.1" 200 None
2026-04-01 06:11:31 [autobid.de] INFO: Scrape type for 3358778: 1
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364506%252F93584966_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364506/93584966_hd.jpg not downloaded yet
2026-04-01 06:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365064%27 HTTP/1.1" 200 None
2026-04-01 06:11:32 [autobid.de] INFO: Scrape type for 3365064: 1
2026-04-01 06:11:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364506%252F93584967_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364506/93584967_hd.jpg not downloaded yet
2026-04-01 06:11:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-passat-variant-2-0-tdi-dsg-business-3365151/details> (referer: None)
2026-04-01 06:11:33 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: b2844702-a938-5846-85ca-c61997ba6478 with identifier: 3365151
2026-04-01 06:11:33 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b2844702-a938-5846-85ca-c61997ba6478 with scrape type 1
2026-04-01 06:11:33 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item b2844702-a938-5846-85ca-c61997ba6478
2026-04-01 06:11:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33652%252F3365151%252F93647275_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33652/3365151/93647275_hd.jpg not downloaded yet
2026-04-01 06:11:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33652%252F3365151%252F93647276_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33652/3365151/93647276_hd.jpg not downloaded yet
2026-04-01 06:11:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33652%252F3365151%252F93647277_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33652/3365151/93647277_hd.jpg not downloaded yet
2026-04-01 06:11:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33652%252F3365151%252F93647278_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33652/3365151/93647278_hd.jpg not downloaded yet
2026-04-01 06:11:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33652%252F3365151%252F93647279_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:11:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33652/3365151/93647279_hd.jpg not downloaded yet
2026-04-01 06:11:36 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=10> (referer: None)
2026-04-01 06:11:36 [autobid.de] INFO: Page 10/145: found 20 items
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366340%27 HTTP/1.1" 200 None
2026-04-01 06:11:36 [autobid.de] INFO: Scrape type for 3366340: 1
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364756%27 HTTP/1.1" 200 None
2026-04-01 06:11:36 [autobid.de] INFO: Scrape type for 3364756: 1
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364754%27 HTTP/1.1" 200 None
2026-04-01 06:11:36 [autobid.de] INFO: Scrape type for 3364754: 1
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365542%27 HTTP/1.1" 200 None
2026-04-01 06:11:36 [autobid.de] INFO: Scrape type for 3365542: 1
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365047%27 HTTP/1.1" 200 None
2026-04-01 06:11:36 [autobid.de] INFO: Scrape type for 3365047: 1
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364755%27 HTTP/1.1" 200 None
2026-04-01 06:11:36 [autobid.de] INFO: Scrape type for 3364755: 1
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365251%27 HTTP/1.1" 200 None
2026-04-01 06:11:36 [autobid.de] INFO: Scrape type for 3365251: 1
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366339%27 HTTP/1.1" 200 None
2026-04-01 06:11:36 [autobid.de] INFO: Scrape type for 3366339: 1
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365156%27 HTTP/1.1" 200 None
2026-04-01 06:11:36 [autobid.de] INFO: Scrape type for 3365156: 1
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365544%27 HTTP/1.1" 200 None
2026-04-01 06:11:36 [autobid.de] INFO: Scrape type for 3365544: 1
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364604%27 HTTP/1.1" 200 None
2026-04-01 06:11:36 [autobid.de] INFO: Scrape type for 3364604: 1
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357570%27 HTTP/1.1" 200 None
2026-04-01 06:11:36 [autobid.de] INFO: Scrape type for 3357570: 1
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365543%27 HTTP/1.1" 200 None
2026-04-01 06:11:36 [autobid.de] INFO: Scrape type for 3365543: 1
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357571%27 HTTP/1.1" 200 None
2026-04-01 06:11:36 [autobid.de] INFO: Scrape type for 3357571: 1
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273349860%27 HTTP/1.1" 200 None
2026-04-01 06:11:36 [autobid.de] INFO: Scrape type for 3349860: 1
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355071%27 HTTP/1.1" 200 None
2026-04-01 06:11:36 [autobid.de] INFO: Scrape type for 3355071: 1
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355810%27 HTTP/1.1" 200 None
2026-04-01 06:11:36 [autobid.de] INFO: Scrape type for 3355810: 1
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357023%27 HTTP/1.1" 200 None
2026-04-01 06:11:36 [autobid.de] INFO: Scrape type for 3357023: 1
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366330%27 HTTP/1.1" 200 None
2026-04-01 06:11:36 [autobid.de] INFO: Scrape type for 3366330: 1
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:11:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366331%27 HTTP/1.1" 200 None
2026-04-01 06:11:36 [autobid.de] INFO: Scrape type for 3366331: 1
2026-04-01 06:11:42 [scrapy.extensions.logstats] INFO: Crawled 97 pages (at 9 pages/min), scraped 1 items (at 1 items/min)
2026-04-01 06:11:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 218MiB
2026-04-01 06:12:42 [scrapy.extensions.logstats] INFO: Crawled 97 pages (at 0 pages/min), scraped 1 items (at 0 items/min)
2026-04-01 06:12:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 218MiB
2026-04-01 06:13:42 [scrapy.extensions.logstats] INFO: Crawled 97 pages (at 0 pages/min), scraped 1 items (at 0 items/min)
2026-04-01 06:13:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 218MiB
2026-04-01 06:14:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item e367fd8b-d9b0-5941-bae6-ce20db337086 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360047/93449626_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:14:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item e367fd8b-d9b0-5941-bae6-ce20db337086 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360047/93449625_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:14:21 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-a6-allroad-quattro-40-tdi-s-tr-led-r-kam-8fach-3366331/details> (referer: None)
2026-04-01 06:14:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 3def9742-7c30-5a8f-8059-9988dd389627 with identifier: 3366331
2026-04-01 06:14:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 3def9742-7c30-5a8f-8059-9988dd389627 with scrape type 1
2026-04-01 06:14:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 3 photos for item 3def9742-7c30-5a8f-8059-9988dd389627
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33664%252F3366331%252F93611779_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:14:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33664/3366331/93611779_hd.jpg not downloaded yet
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33664%252F3366331%252F93611781_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:14:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33664/3366331/93611781_hd.jpg not downloaded yet
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33664%252F3366331%252F93611778_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:14:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33664/3366331/93611778_hd.jpg not downloaded yet
2026-04-01 06:14:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=11> (referer: None)
2026-04-01 06:14:22 [autobid.de] INFO: Page 11/145: found 20 items
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364608%27 HTTP/1.1" 200 None
2026-04-01 06:14:22 [autobid.de] INFO: Scrape type for 3364608: 1
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364609%27 HTTP/1.1" 200 None
2026-04-01 06:14:22 [autobid.de] INFO: Scrape type for 3364609: 1
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366328%27 HTTP/1.1" 200 None
2026-04-01 06:14:22 [autobid.de] INFO: Scrape type for 3366328: 1
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366332%27 HTTP/1.1" 200 None
2026-04-01 06:14:22 [autobid.de] INFO: Scrape type for 3366332: 1
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366333%27 HTTP/1.1" 200 None
2026-04-01 06:14:22 [autobid.de] INFO: Scrape type for 3366333: 1
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366335%27 HTTP/1.1" 200 None
2026-04-01 06:14:22 [autobid.de] INFO: Scrape type for 3366335: 1
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366336%27 HTTP/1.1" 200 None
2026-04-01 06:14:22 [autobid.de] INFO: Scrape type for 3366336: 1
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366341%27 HTTP/1.1" 200 None
2026-04-01 06:14:22 [autobid.de] INFO: Scrape type for 3366341: 1
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366369%27 HTTP/1.1" 200 None
2026-04-01 06:14:22 [autobid.de] INFO: Scrape type for 3366369: 1
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366376%27 HTTP/1.1" 200 None
2026-04-01 06:14:22 [autobid.de] INFO: Scrape type for 3366376: 1
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366371%27 HTTP/1.1" 200 None
2026-04-01 06:14:22 [autobid.de] INFO: Scrape type for 3366371: 1
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366337%27 HTTP/1.1" 200 None
2026-04-01 06:14:22 [autobid.de] INFO: Scrape type for 3366337: 1
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366342%27 HTTP/1.1" 200 None
2026-04-01 06:14:22 [autobid.de] INFO: Scrape type for 3366342: 1
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366343%27 HTTP/1.1" 200 None
2026-04-01 06:14:22 [autobid.de] INFO: Scrape type for 3366343: 1
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366344%27 HTTP/1.1" 200 None
2026-04-01 06:14:22 [autobid.de] INFO: Scrape type for 3366344: 1
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366346%27 HTTP/1.1" 200 None
2026-04-01 06:14:22 [autobid.de] INFO: Scrape type for 3366346: 1
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366370%27 HTTP/1.1" 200 None
2026-04-01 06:14:22 [autobid.de] INFO: Scrape type for 3366370: 1
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366345%27 HTTP/1.1" 200 None
2026-04-01 06:14:22 [autobid.de] INFO: Scrape type for 3366345: 1
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366347%27 HTTP/1.1" 200 None
2026-04-01 06:14:22 [autobid.de] INFO: Scrape type for 3366347: 1
2026-04-01 06:14:22 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item e367fd8b-d9b0-5941-bae6-ce20db337086 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360047/93449624_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358299%27 HTTP/1.1" 200 None
2026-04-01 06:14:22 [autobid.de] INFO: Scrape type for 3358299: 1
2026-04-01 06:14:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/seat-ateca-xcellence-2-0-tdi-4drive-dsg-8fach-3366330/details> (referer: None)
2026-04-01 06:14:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: e66f4a54-8d5e-5443-b5f9-e02d030fcd52 with identifier: 3366330
2026-04-01 06:14:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e66f4a54-8d5e-5443-b5f9-e02d030fcd52 with scrape type 1
2026-04-01 06:14:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 3 photos for item e66f4a54-8d5e-5443-b5f9-e02d030fcd52
2026-04-01 06:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33664%252F3366330%252F93611775_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:14:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33664/3366330/93611775_hd.jpg not downloaded yet
2026-04-01 06:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33664%252F3366330%252F93611772_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:14:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33664/3366330/93611772_hd.jpg not downloaded yet
2026-04-01 06:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33664%252F3366330%252F93611773_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:14:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33664/3366330/93611773_hd.jpg not downloaded yet
2026-04-01 06:14:24 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: e367fd8b-d9b0-5941-bae6-ce20db337086 with identifier: 3360047
2026-04-01 06:14:24 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 06:14:24 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=silver+Metallic HTTP/1.1" 200 None
2026-04-01 06:14:24 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 06:14:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 06:14:24 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 06:14:24 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=Petrol HTTP/1.1" 200 None
2026-04-01 06:14:24 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 06:14:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 06:14:24 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: e367fd8b-d9b0-5941-bae6-ce20db337086 with identifier: 3360047
2026-04-01 06:14:24 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: e367fd8b-d9b0-5941-bae6-ce20db337086 with identifier: 3360047 to the API
2026-04-01 06:14:24 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 06:14:24 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 06:14:24 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360047/93449623_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 81210',
'body_type': 'Saloon',
'category': 'Saloon',
'color': 'silver Metallic',
'currency': 'EUR',
'engine': '1.4L',
'engine_horse_power': 86,
'fuel_type': 'Petrol',
'id': 'e367fd8b-d9b0-5941-bae6-ce20db337086',
'is_damaged': False,
'is_operable': True,
'is_vat_deductible': False,
'link_to_web_offer': 'https://autobid.de/en/item/skoda-fabia-1-4-3360047',
'listing': 'auction',
'location': 'Braunschweig',
'make': 'Skoda',
'model': 'Fabia',
'odometer': 88200,
'odometer_reading_unit': 'km',
'origin_country_code': 'DE',
'original_photo_count': 5,
'photo_urls': ['https://cdn.autobid.de/data/cars/33601/3360047/93449623_hd.jpg',
'https://cdn.autobid.de/data/cars/33601/3360047/93449624_hd.jpg',
'https://cdn.autobid.de/data/cars/33601/3360047/93449625_hd.jpg',
'https://cdn.autobid.de/data/cars/33601/3360047/93449626_hd.jpg',
'https://cdn.autobid.de/data/cars/33601/3360047/93449627_hd.jpg'],
'photos': [],
'price': 4100,
'price_includes_vat': False,
'registration_date': '2014-07-01',
'seats_number': 5,
'seller_name': 'Subsidiary/authorised dealer of the brand(s): Skoda',
'title': 'Skoda Fabia 1,4',
'transmission': 'manual',
'vin': None,
'year': 2014}
2026-04-01 06:14:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360047%27 HTTP/1.1" 200 None
2026-04-01 06:14:24 [autobid.de] INFO: Saving data for 3360047: {'created_time': 1775024064.907253, 'last_price_update_time': 1775024064.907268}
2026-04-01 06:14:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3360047') HTTP/1.1" 204 0
2026-04-01 06:14:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item e367fd8b-d9b0-5941-bae6-ce20db337086 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360047/93449623_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:14:25 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-caddy-maxi-cargo-2-0-tdi-dsg-3366337/details> (referer: None)
2026-04-01 06:14:25 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: b287c046-38b8-5065-a811-ed1f6106a967 with identifier: 3366337
2026-04-01 06:14:25 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b287c046-38b8-5065-a811-ed1f6106a967 with scrape type 1
2026-04-01 06:14:25 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 3 photos for item b287c046-38b8-5065-a811-ed1f6106a967
2026-04-01 06:14:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33664%252F3366337%252F93611867_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:14:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33664/3366337/93611867_hd.jpg not downloaded yet
2026-04-01 06:14:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33664%252F3366337%252F93611866_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:14:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33664/3366337/93611866_hd.jpg not downloaded yet
2026-04-01 06:14:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33664%252F3366337%252F93611865_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:14:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33664/3366337/93611865_hd.jpg not downloaded yet
2026-04-01 06:14:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 76bcaa28-7f4d-5479-87ca-862563a68323 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33528/3352727/93505328_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:14:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 76bcaa28-7f4d-5479-87ca-862563a68323 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33528/3352727/93505342_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:14:28 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=12> (referer: None)
2026-04-01 06:14:28 [autobid.de] INFO: Page 12/145: found 20 items
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359755%27 HTTP/1.1" 200 None
2026-04-01 06:14:28 [autobid.de] INFO: Scrape type for 3359755: 1
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360056%27 HTTP/1.1" 200 None
2026-04-01 06:14:28 [autobid.de] INFO: Scrape type for 3360056: 1
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355058%27 HTTP/1.1" 200 None
2026-04-01 06:14:28 [autobid.de] INFO: Scrape type for 3355058: 1
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357000%27 HTTP/1.1" 200 None
2026-04-01 06:14:28 [autobid.de] INFO: Scrape type for 3357000: 1
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356997%27 HTTP/1.1" 200 None
2026-04-01 06:14:28 [autobid.de] INFO: Scrape type for 3356997: 1
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356998%27 HTTP/1.1" 200 None
2026-04-01 06:14:28 [autobid.de] INFO: Scrape type for 3356998: 1
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356999%27 HTTP/1.1" 200 None
2026-04-01 06:14:28 [autobid.de] INFO: Scrape type for 3356999: 1
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357001%27 HTTP/1.1" 200 None
2026-04-01 06:14:28 [autobid.de] INFO: Scrape type for 3357001: 1
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357353%27 HTTP/1.1" 200 None
2026-04-01 06:14:28 [autobid.de] INFO: Scrape type for 3357353: 1
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361285%27 HTTP/1.1" 200 None
2026-04-01 06:14:28 [autobid.de] INFO: Scrape type for 3361285: 1
2026-04-01 06:14:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item e1bc0520-5dc6-5486-9d07-248a98581ca8 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33597/3359677/93637519_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361286%27 HTTP/1.1" 200 None
2026-04-01 06:14:28 [autobid.de] INFO: Scrape type for 3361286: 1
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361287%27 HTTP/1.1" 200 None
2026-04-01 06:14:28 [autobid.de] INFO: Scrape type for 3361287: 1
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361288%27 HTTP/1.1" 200 None
2026-04-01 06:14:28 [autobid.de] INFO: Scrape type for 3361288: 1
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361289%27 HTTP/1.1" 200 None
2026-04-01 06:14:28 [autobid.de] INFO: Scrape type for 3361289: 1
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361290%27 HTTP/1.1" 200 None
2026-04-01 06:14:28 [autobid.de] INFO: Scrape type for 3361290: 1
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361291%27 HTTP/1.1" 200 None
2026-04-01 06:14:28 [autobid.de] INFO: Scrape type for 3361291: 1
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361292%27 HTTP/1.1" 200 None
2026-04-01 06:14:28 [autobid.de] INFO: Scrape type for 3361292: 1
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361293%27 HTTP/1.1" 200 None
2026-04-01 06:14:28 [autobid.de] INFO: Scrape type for 3361293: 1
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361294%27 HTTP/1.1" 200 None
2026-04-01 06:14:28 [autobid.de] INFO: Scrape type for 3361294: 1
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361295%27 HTTP/1.1" 200 None
2026-04-01 06:14:28 [autobid.de] INFO: Scrape type for 3361295: 1
2026-04-01 06:14:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/bmw-x3-xdrive-20d-steptronic-xline-3357000/details> (referer: None)
2026-04-01 06:14:29 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 50331e53-0515-52a0-88c2-53ac6ac77087 with identifier: 3357000
2026-04-01 06:14:29 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 50331e53-0515-52a0-88c2-53ac6ac77087 with scrape type 1
2026-04-01 06:14:29 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 50331e53-0515-52a0-88c2-53ac6ac77087
2026-04-01 06:14:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33571%252F3357000%252F93485308_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:14:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33571/3357000/93485308_hd.jpg not downloaded yet
2026-04-01 06:14:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33571%252F3357000%252F93485307_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:14:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33571/3357000/93485307_hd.jpg not downloaded yet
2026-04-01 06:14:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33571%252F3357000%252F93485300_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:14:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33571/3357000/93485300_hd.jpg not downloaded yet
2026-04-01 06:14:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33571%252F3357000%252F93485304_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:14:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33571/3357000/93485304_hd.jpg not downloaded yet
2026-04-01 06:14:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33571%252F3357000%252F93485305_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:14:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33571/3357000/93485305_hd.jpg not downloaded yet
2026-04-01 06:14:29 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 0266a864-f637-53c1-a82a-f69a264e327c due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360050/93448779_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:14:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=13> (referer: None)
2026-04-01 06:14:31 [autobid.de] INFO: Page 13/145: found 20 items
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361296%27 HTTP/1.1" 200 None
2026-04-01 06:14:31 [autobid.de] INFO: Scrape type for 3361296: 1
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361297%27 HTTP/1.1" 200 None
2026-04-01 06:14:31 [autobid.de] INFO: Scrape type for 3361297: 1
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361298%27 HTTP/1.1" 200 None
2026-04-01 06:14:31 [autobid.de] INFO: Scrape type for 3361298: 1
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361299%27 HTTP/1.1" 200 None
2026-04-01 06:14:31 [autobid.de] INFO: Scrape type for 3361299: 1
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361300%27 HTTP/1.1" 200 None
2026-04-01 06:14:31 [autobid.de] INFO: Scrape type for 3361300: 1
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361301%27 HTTP/1.1" 200 None
2026-04-01 06:14:31 [autobid.de] INFO: Scrape type for 3361301: 1
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361302%27 HTTP/1.1" 200 None
2026-04-01 06:14:31 [autobid.de] INFO: Scrape type for 3361302: 1
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361303%27 HTTP/1.1" 200 None
2026-04-01 06:14:31 [autobid.de] INFO: Scrape type for 3361303: 1
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361304%27 HTTP/1.1" 200 None
2026-04-01 06:14:31 [autobid.de] INFO: Scrape type for 3361304: 1
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361305%27 HTTP/1.1" 200 None
2026-04-01 06:14:31 [autobid.de] INFO: Scrape type for 3361305: 1
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361306%27 HTTP/1.1" 200 None
2026-04-01 06:14:31 [autobid.de] INFO: Scrape type for 3361306: 1
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361307%27 HTTP/1.1" 200 None
2026-04-01 06:14:31 [autobid.de] INFO: Scrape type for 3361307: 1
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361308%27 HTTP/1.1" 200 None
2026-04-01 06:14:31 [autobid.de] INFO: Scrape type for 3361308: 1
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361309%27 HTTP/1.1" 200 None
2026-04-01 06:14:31 [autobid.de] INFO: Scrape type for 3361309: 1
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361310%27 HTTP/1.1" 200 None
2026-04-01 06:14:31 [autobid.de] INFO: Scrape type for 3361310: 1
2026-04-01 06:14:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362455%27 HTTP/1.1" 200 None
2026-04-01 06:14:32 [autobid.de] INFO: Scrape type for 3362455: 1
2026-04-01 06:14:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362995%27 HTTP/1.1" 200 None
2026-04-01 06:14:32 [autobid.de] INFO: Scrape type for 3362995: 1
2026-04-01 06:14:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363011%27 HTTP/1.1" 200 None
2026-04-01 06:14:32 [autobid.de] INFO: Scrape type for 3363011: 1
2026-04-01 06:14:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363701%27 HTTP/1.1" 200 None
2026-04-01 06:14:32 [autobid.de] INFO: Scrape type for 3363701: 1
2026-04-01 06:14:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:14:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364667%27 HTTP/1.1" 200 None
2026-04-01 06:14:32 [autobid.de] INFO: Scrape type for 3364667: 1
2026-04-01 06:14:42 [scrapy.extensions.logstats] INFO: Crawled 104 pages (at 7 pages/min), scraped 2 items (at 1 items/min)
2026-04-01 06:14:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 218MiB
2026-04-01 06:15:42 [scrapy.extensions.logstats] INFO: Crawled 104 pages (at 0 pages/min), scraped 2 items (at 0 items/min)
2026-04-01 06:15:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 218MiB
2026-04-01 06:16:42 [scrapy.extensions.logstats] INFO: Crawled 104 pages (at 0 pages/min), scraped 2 items (at 0 items/min)
2026-04-01 06:16:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 218MiB
2026-04-01 06:17:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 1b91743c-5ad9-5357-8f18-cf24c4119a5b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33527/3352621/93438793_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:17:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 066652a8-7812-57ae-ab95-81add126dea4 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33576/3357581/93454480_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:17:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-golf-1-0-tsi-bmt-comfortline-3364667/details> (referer: None)
2026-04-01 06:17:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: be7412ef-1daf-5fba-9e38-d0ee1cd0e50a with identifier: 3364667
2026-04-01 06:17:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item be7412ef-1daf-5fba-9e38-d0ee1cd0e50a with scrape type 1
2026-04-01 06:17:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item be7412ef-1daf-5fba-9e38-d0ee1cd0e50a
2026-04-01 06:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364667%252F93542709_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:17:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364667/93542709_hd.jpg not downloaded yet
2026-04-01 06:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364667%252F93542698_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:17:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364667/93542698_hd.jpg not downloaded yet
2026-04-01 06:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364667%252F93542708_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:17:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364667/93542708_hd.jpg not downloaded yet
2026-04-01 06:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364667%252F93542696_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:17:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364667/93542696_hd.jpg not downloaded yet
2026-04-01 06:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364667%252F93542711_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:17:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364667/93542711_hd.jpg not downloaded yet
2026-04-01 06:17:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=14> (referer: None)
2026-04-01 06:17:22 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 066652a8-7812-57ae-ab95-81add126dea4 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33576/3357581/93454479_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:17:22 [autobid.de] INFO: Page 14/145: found 20 items
2026-04-01 06:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345123%27 HTTP/1.1" 200 None
2026-04-01 06:17:22 [autobid.de] INFO: Scrape type for 3345123: 1
2026-04-01 06:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345125%27 HTTP/1.1" 200 None
2026-04-01 06:17:22 [autobid.de] INFO: Scrape type for 3345125: 1
2026-04-01 06:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359075%27 HTTP/1.1" 200 None
2026-04-01 06:17:22 [autobid.de] INFO: Scrape type for 3359075: 1
2026-04-01 06:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359076%27 HTTP/1.1" 200 None
2026-04-01 06:17:22 [autobid.de] INFO: Scrape type for 3359076: 1
2026-04-01 06:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359077%27 HTTP/1.1" 200 None
2026-04-01 06:17:22 [autobid.de] INFO: Scrape type for 3359077: 1
2026-04-01 06:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345121%27 HTTP/1.1" 200 None
2026-04-01 06:17:22 [autobid.de] INFO: Scrape type for 3345121: 1
2026-04-01 06:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345126%27 HTTP/1.1" 200 None
2026-04-01 06:17:23 [autobid.de] INFO: Scrape type for 3345126: 1
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365053%27 HTTP/1.1" 200 None
2026-04-01 06:17:23 [autobid.de] INFO: Scrape type for 3365053: 1
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362878%27 HTTP/1.1" 200 None
2026-04-01 06:17:23 [autobid.de] INFO: Scrape type for 3362878: 1
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365130%27 HTTP/1.1" 200 None
2026-04-01 06:17:23 [autobid.de] INFO: Scrape type for 3365130: 1
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359994%27 HTTP/1.1" 200 None
2026-04-01 06:17:23 [autobid.de] INFO: Scrape type for 3359994: 1
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359995%27 HTTP/1.1" 200 None
2026-04-01 06:17:23 [autobid.de] INFO: Scrape type for 3359995: 1
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273335128%27 HTTP/1.1" 200 None
2026-04-01 06:17:23 [autobid.de] INFO: Scrape type for 3335128: 1
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273341001%27 HTTP/1.1" 200 None
2026-04-01 06:17:23 [autobid.de] INFO: Scrape type for 3341001: 1
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273346814%27 HTTP/1.1" 200 None
2026-04-01 06:17:23 [autobid.de] INFO: Scrape type for 3346814: 1
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361895%27 HTTP/1.1" 200 None
2026-04-01 06:17:23 [autobid.de] INFO: Scrape type for 3361895: 1
2026-04-01 06:17:23 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item e66f4a54-8d5e-5443-b5f9-e02d030fcd52 due to An error occurred while connecting: 104: Connection reset by peer.
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361896%27 HTTP/1.1" 200 None
2026-04-01 06:17:23 [autobid.de] INFO: Scrape type for 3361896: 1
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357584%27 HTTP/1.1" 200 None
2026-04-01 06:17:23 [autobid.de] INFO: Scrape type for 3357584: 1
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361693%27 HTTP/1.1" 200 None
2026-04-01 06:17:23 [autobid.de] INFO: Scrape type for 3361693: 1
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361694%27 HTTP/1.1" 200 None
2026-04-01 06:17:23 [autobid.de] INFO: Scrape type for 3361694: 1
2026-04-01 06:17:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/lexus-ls500h-awd-3363701/details> (referer: None)
2026-04-01 06:17:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 5c939d81-707d-5404-b3e3-79590a8cad81 with identifier: 3363701
2026-04-01 06:17:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 5c939d81-707d-5404-b3e3-79590a8cad81 with scrape type 1
2026-04-01 06:17:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 5c939d81-707d-5404-b3e3-79590a8cad81
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33638%252F3363701%252F93533503_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:17:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33638/3363701/93533503_hd.jpg not downloaded yet
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33638%252F3363701%252F93533527_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:17:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33638/3363701/93533527_hd.jpg not downloaded yet
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33638%252F3363701%252F93533511_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:17:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33638/3363701/93533511_hd.jpg not downloaded yet
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33638%252F3363701%252F93533516_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:17:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33638/3363701/93533516_hd.jpg not downloaded yet
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33638%252F3363701%252F93533526_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:17:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33638/3363701/93533526_hd.jpg not downloaded yet
2026-04-01 06:17:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-passat-variant-1-4-tsi-bmt-comfortline-3365130/details> (referer: None)
2026-04-01 06:17:24 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 70745f8f-d834-5336-91d3-7e3f98559def with identifier: 3365130
2026-04-01 06:17:24 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 70745f8f-d834-5336-91d3-7e3f98559def with scrape type 1
2026-04-01 06:17:24 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 70745f8f-d834-5336-91d3-7e3f98559def
2026-04-01 06:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33652%252F3365130%252F93563389_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:17:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33652/3365130/93563389_hd.jpg not downloaded yet
2026-04-01 06:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33652%252F3365130%252F93563382_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:17:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33652/3365130/93563382_hd.jpg not downloaded yet
2026-04-01 06:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33652%252F3365130%252F93563371_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:17:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33652/3365130/93563371_hd.jpg not downloaded yet
2026-04-01 06:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33652%252F3365130%252F93563374_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:17:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33652/3365130/93563374_hd.jpg not downloaded yet
2026-04-01 06:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33652%252F3365130%252F93563384_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:17:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33652/3365130/93563384_hd.jpg not downloaded yet
2026-04-01 06:17:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 0c0d54f5-ec5f-5941-951b-b39e63761ed8 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362689/93483276_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:17:25 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-tiguan-1-4-tsi-bmt-dsg-sound-3361694/details> (referer: None)
2026-04-01 06:17:25 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 07299513-4d2c-5e69-bfc3-ca4ca75c839f with identifier: 3361694
2026-04-01 06:17:25 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 07299513-4d2c-5e69-bfc3-ca4ca75c839f with scrape type 1
2026-04-01 06:17:25 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 07299513-4d2c-5e69-bfc3-ca4ca75c839f
2026-04-01 06:17:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33617%252F3361694%252F93586431_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:17:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33617/3361694/93586431_hd.jpg not downloaded yet
2026-04-01 06:17:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33617%252F3361694%252F93586432_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:17:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33617/3361694/93586432_hd.jpg not downloaded yet
2026-04-01 06:17:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33617%252F3361694%252F93586433_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:17:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33617/3361694/93586433_hd.jpg not downloaded yet
2026-04-01 06:17:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33617%252F3361694%252F93586434_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:17:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33617/3361694/93586434_hd.jpg not downloaded yet
2026-04-01 06:17:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:17:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33617%252F3361694%252F93586435_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:17:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33617/3361694/93586435_hd.jpg not downloaded yet
2026-04-01 06:17:26 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 0c0d54f5-ec5f-5941-951b-b39e63761ed8 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362689/93483280_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:17:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 0c0d54f5-ec5f-5941-951b-b39e63761ed8 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362689/93483279_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:17:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 0c0d54f5-ec5f-5941-951b-b39e63761ed8 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362689/93483278_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:17:29 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item b2844702-a938-5846-85ca-c61997ba6478 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33652/3365151/93647279_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:17:42 [scrapy.extensions.logstats] INFO: Crawled 109 pages (at 5 pages/min), scraped 2 items (at 0 items/min)
2026-04-01 06:17:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 218MiB
2026-04-01 06:18:42 [scrapy.extensions.logstats] INFO: Crawled 109 pages (at 0 pages/min), scraped 2 items (at 0 items/min)
2026-04-01 06:18:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 218MiB
2026-04-01 06:19:42 [scrapy.extensions.logstats] INFO: Crawled 109 pages (at 0 pages/min), scraped 2 items (at 0 items/min)
2026-04-01 06:19:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 218MiB
2026-04-01 06:20:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 3def9742-7c30-5a8f-8059-9988dd389627 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33664/3366331/93611778_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:20:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item e66f4a54-8d5e-5443-b5f9-e02d030fcd52 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33664/3366330/93611773_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:20:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item b287c046-38b8-5065-a811-ed1f6106a967 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33664/3366337/93611865_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:20:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item b287c046-38b8-5065-a811-ed1f6106a967 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33664/3366337/93611866_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:20:27 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: b287c046-38b8-5065-a811-ed1f6106a967 with identifier: 3366337
2026-04-01 06:20:27 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 06:20:27 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=grey HTTP/1.1" 200 None
2026-04-01 06:20:27 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 06:20:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 06:20:27 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: b287c046-38b8-5065-a811-ed1f6106a967 with identifier: 3366337
2026-04-01 06:20:27 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: b287c046-38b8-5065-a811-ed1f6106a967 with identifier: 3366337 to the API
2026-04-01 06:20:27 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 06:20:27 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 06:20:27 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33664/3366337/93611867_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 81309',
'body_type': 'Light duty truck',
'category': 'Light duty truck',
'color': 'grey',
'currency': 'EUR',
'emission_standard': 'EURO 6d',
'engine': '2.0L',
'engine_horse_power': 122,
'expiration_date': '2026-04-01T08:00:00Z',
'fuel_type': 'Diesel',
'id': 'b287c046-38b8-5065-a811-ed1f6106a967',
'is_damaged': True,
'is_operable': True,
'is_vat_included': True,
'link_to_web_offer': 'https://autobid.de/en/item/vw-caddy-maxi-cargo-2-0-tdi-dsg-3366337',
'listing': 'auction',
'location': 'Fürstenfeldbruck',
'make': 'VW',
'model': 'Caddy Maxi Cargo',
'odometer': 101200,
'odometer_reading_unit': 'km',
'origin_country_code': 'DE',
'original_photo_count': 3,
'photo_urls': ['https://cdn.autobid.de/data/cars/33664/3366337/93611867_hd.jpg',
'https://cdn.autobid.de/data/cars/33664/3366337/93611866_hd.jpg',
'https://cdn.autobid.de/data/cars/33664/3366337/93611865_hd.jpg'],
'photos': [],
'price': 13700,
'price_includes_vat': True,
'registration_date': '2023-03-01',
'seller_name': 'Subsidiary/authorised dealer of the brand(s): Audi',
'title': 'VW Caddy Maxi Cargo 2.0 TDI DSG',
'transmission': 'automatic',
'vin': None,
'year': 2023}
2026-04-01 06:20:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:20:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366337%27 HTTP/1.1" 200 None
2026-04-01 06:20:27 [autobid.de] INFO: Saving data for 3366337: {'created_time': 1775024427.781558, 'last_price_update_time': 1775024427.781572, 'auction_closing_time': 1775030400.0}
2026-04-01 06:20:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:20:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3366337') HTTP/1.1" 204 0
2026-04-01 06:20:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item b287c046-38b8-5065-a811-ed1f6106a967 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33664/3366337/93611867_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:20:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 50331e53-0515-52a0-88c2-53ac6ac77087 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33571/3357000/93485305_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:20:29 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 50331e53-0515-52a0-88c2-53ac6ac77087 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33571/3357000/93485304_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:20:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item be7412ef-1daf-5fba-9e38-d0ee1cd0e50a due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33647/3364667/93542711_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:20:42 [scrapy.extensions.logstats] INFO: Crawled 109 pages (at 0 pages/min), scraped 3 items (at 1 items/min)
2026-04-01 06:20:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 218MiB
2026-04-01 06:21:42 [scrapy.extensions.logstats] INFO: Crawled 109 pages (at 0 pages/min), scraped 3 items (at 0 items/min)
2026-04-01 06:21:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 218MiB
2026-04-01 06:22:42 [scrapy.extensions.logstats] INFO: Crawled 109 pages (at 0 pages/min), scraped 3 items (at 0 items/min)
2026-04-01 06:22:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 218MiB
2026-04-01 06:23:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item be7412ef-1daf-5fba-9e38-d0ee1cd0e50a due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33647/3364667/93542696_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:23:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 5c939d81-707d-5404-b3e3-79590a8cad81 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33638/3363701/93533526_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:23:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-q5-2-0-tdi-quattro-3361693/details> (referer: None)
2026-04-01 06:23:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 741026e9-3931-53d3-8e0a-bc0ab3c159d4 with identifier: 3361693
2026-04-01 06:23:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 741026e9-3931-53d3-8e0a-bc0ab3c159d4 with scrape type 1
2026-04-01 06:23:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 741026e9-3931-53d3-8e0a-bc0ab3c159d4
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33617%252F3361693%252F93582830_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33617/3361693/93582830_hd.jpg not downloaded yet
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33617%252F3361693%252F93582831_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33617/3361693/93582831_hd.jpg not downloaded yet
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33617%252F3361693%252F93582832_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33617/3361693/93582832_hd.jpg not downloaded yet
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33617%252F3361693%252F93582833_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33617/3361693/93582833_hd.jpg not downloaded yet
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33617%252F3361693%252F93582834_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33617/3361693/93582834_hd.jpg not downloaded yet
2026-04-01 06:23:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=15> (referer: None)
2026-04-01 06:23:22 [autobid.de] INFO: Page 15/145: found 20 items
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361696%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [autobid.de] INFO: Scrape type for 3361696: 1
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360563%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [autobid.de] INFO: Scrape type for 3360563: 1
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364673%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [autobid.de] INFO: Scrape type for 3364673: 1
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364674%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [autobid.de] INFO: Scrape type for 3364674: 1
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364675%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [autobid.de] INFO: Scrape type for 3364675: 1
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364676%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [autobid.de] INFO: Scrape type for 3364676: 1
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364677%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [autobid.de] INFO: Scrape type for 3364677: 1
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364678%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [autobid.de] INFO: Scrape type for 3364678: 1
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364679%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [autobid.de] INFO: Scrape type for 3364679: 1
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364680%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [autobid.de] INFO: Scrape type for 3364680: 1
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364681%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [autobid.de] INFO: Scrape type for 3364681: 1
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364682%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [autobid.de] INFO: Scrape type for 3364682: 1
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364683%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [autobid.de] INFO: Scrape type for 3364683: 1
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364684%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [autobid.de] INFO: Scrape type for 3364684: 1
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364685%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [autobid.de] INFO: Scrape type for 3364685: 1
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364707%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [autobid.de] INFO: Scrape type for 3364707: 1
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364687%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [autobid.de] INFO: Scrape type for 3364687: 1
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365073%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [autobid.de] INFO: Scrape type for 3365073: 1
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365074%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [autobid.de] INFO: Scrape type for 3365074: 1
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365075%27 HTTP/1.1" 200 None
2026-04-01 06:23:22 [autobid.de] INFO: Scrape type for 3365075: 1
2026-04-01 06:23:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-a3-sportback-1-4-16v-tfsi-ambition-3357584/details> (referer: None)
2026-04-01 06:23:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 881d9bea-9951-513f-8a14-27e5bc036d7a with identifier: 3357584
2026-04-01 06:23:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 881d9bea-9951-513f-8a14-27e5bc036d7a with scrape type 1
2026-04-01 06:23:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 881d9bea-9951-513f-8a14-27e5bc036d7a
2026-04-01 06:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357584%252F93583197_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:23:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357584/93583197_hd.jpg not downloaded yet
2026-04-01 06:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357584%252F93583198_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:23:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357584/93583198_hd.jpg not downloaded yet
2026-04-01 06:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357584%252F93583203_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:23:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357584/93583203_hd.jpg not downloaded yet
2026-04-01 06:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357584%252F93583204_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:23:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357584/93583204_hd.jpg not downloaded yet
2026-04-01 06:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357584%252F93583202_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:23:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357584/93583202_hd.jpg not downloaded yet
2026-04-01 06:23:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 70745f8f-d834-5336-91d3-7e3f98559def due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33652/3365130/93563384_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:23:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 70745f8f-d834-5336-91d3-7e3f98559def due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33652/3365130/93563374_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:23:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=16> (referer: None)
2026-04-01 06:23:26 [autobid.de] INFO: Page 16/145: found 20 items
2026-04-01 06:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365076%27 HTTP/1.1" 200 None
2026-04-01 06:23:26 [autobid.de] INFO: Scrape type for 3365076: 1
2026-04-01 06:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365077%27 HTTP/1.1" 200 None
2026-04-01 06:23:26 [autobid.de] INFO: Scrape type for 3365077: 1
2026-04-01 06:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365078%27 HTTP/1.1" 200 None
2026-04-01 06:23:26 [autobid.de] INFO: Scrape type for 3365078: 1
2026-04-01 06:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365079%27 HTTP/1.1" 200 None
2026-04-01 06:23:26 [autobid.de] INFO: Scrape type for 3365079: 1
2026-04-01 06:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364686%27 HTTP/1.1" 200 None
2026-04-01 06:23:26 [autobid.de] INFO: Scrape type for 3364686: 1
2026-04-01 06:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365080%27 HTTP/1.1" 200 None
2026-04-01 06:23:26 [autobid.de] INFO: Scrape type for 3365080: 1
2026-04-01 06:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365081%27 HTTP/1.1" 200 None
2026-04-01 06:23:26 [autobid.de] INFO: Scrape type for 3365081: 1
2026-04-01 06:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365084%27 HTTP/1.1" 200 None
2026-04-01 06:23:26 [autobid.de] INFO: Scrape type for 3365084: 1
2026-04-01 06:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365085%27 HTTP/1.1" 200 None
2026-04-01 06:23:26 [autobid.de] INFO: Scrape type for 3365085: 1
2026-04-01 06:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365565%27 HTTP/1.1" 200 None
2026-04-01 06:23:26 [autobid.de] INFO: Scrape type for 3365565: 1
2026-04-01 06:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365566%27 HTTP/1.1" 200 None
2026-04-01 06:23:27 [autobid.de] INFO: Scrape type for 3365566: 1
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365567%27 HTTP/1.1" 200 None
2026-04-01 06:23:27 [autobid.de] INFO: Scrape type for 3365567: 1
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365568%27 HTTP/1.1" 200 None
2026-04-01 06:23:27 [autobid.de] INFO: Scrape type for 3365568: 1
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365569%27 HTTP/1.1" 200 None
2026-04-01 06:23:27 [autobid.de] INFO: Scrape type for 3365569: 1
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360560%27 HTTP/1.1" 200 None
2026-04-01 06:23:27 [autobid.de] INFO: Scrape type for 3360560: 1
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360562%27 HTTP/1.1" 200 None
2026-04-01 06:23:27 [autobid.de] INFO: Scrape type for 3360562: 1
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360564%27 HTTP/1.1" 200 None
2026-04-01 06:23:27 [autobid.de] INFO: Scrape type for 3360564: 1
2026-04-01 06:23:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 07299513-4d2c-5e69-bfc3-ca4ca75c839f due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33617/3361694/93586435_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360549%27 HTTP/1.1" 200 None
2026-04-01 06:23:27 [autobid.de] INFO: Scrape type for 3360549: 1
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360550%27 HTTP/1.1" 200 None
2026-04-01 06:23:27 [autobid.de] INFO: Scrape type for 3360550: 1
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360553%27 HTTP/1.1" 200 None
2026-04-01 06:23:27 [autobid.de] INFO: Scrape type for 3360553: 1
2026-04-01 06:23:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/citroen-c4-grand-picasso-2-0-bluehdi-executive-3365567/details> (referer: None)
2026-04-01 06:23:27 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 54fe1c0c-3e8f-5551-ba9f-9b5bd3df09e8 with identifier: 3365567
2026-04-01 06:23:27 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 54fe1c0c-3e8f-5551-ba9f-9b5bd3df09e8 with scrape type 1
2026-04-01 06:23:27 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 54fe1c0c-3e8f-5551-ba9f-9b5bd3df09e8
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365567%252F93586046_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:23:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365567/93586046_hd.jpg not downloaded yet
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365567%252F93586063_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:23:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365567/93586063_hd.jpg not downloaded yet
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365567%252F93586065_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:23:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365567/93586065_hd.jpg not downloaded yet
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365567%252F93586050_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:23:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365567/93586050_hd.jpg not downloaded yet
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365567%252F93586069_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:23:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365567/93586069_hd.jpg not downloaded yet
2026-04-01 06:23:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 07299513-4d2c-5e69-bfc3-ca4ca75c839f due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33617/3361694/93586434_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:23:29 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 07299513-4d2c-5e69-bfc3-ca4ca75c839f due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33617/3361694/93586433_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:23:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=17> (referer: None)
2026-04-01 06:23:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 07299513-4d2c-5e69-bfc3-ca4ca75c839f due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33617/3361694/93586432_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:23:30 [autobid.de] INFO: Page 17/145: found 20 items
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360551%27 HTTP/1.1" 200 None
2026-04-01 06:23:30 [autobid.de] INFO: Scrape type for 3360551: 1
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360552%27 HTTP/1.1" 200 None
2026-04-01 06:23:30 [autobid.de] INFO: Scrape type for 3360552: 1
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360556%27 HTTP/1.1" 200 None
2026-04-01 06:23:30 [autobid.de] INFO: Scrape type for 3360556: 1
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360554%27 HTTP/1.1" 200 None
2026-04-01 06:23:30 [autobid.de] INFO: Scrape type for 3360554: 1
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360555%27 HTTP/1.1" 200 None
2026-04-01 06:23:30 [autobid.de] INFO: Scrape type for 3360555: 1
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360557%27 HTTP/1.1" 200 None
2026-04-01 06:23:30 [autobid.de] INFO: Scrape type for 3360557: 1
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360561%27 HTTP/1.1" 200 None
2026-04-01 06:23:30 [autobid.de] INFO: Scrape type for 3360561: 1
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361697%27 HTTP/1.1" 200 None
2026-04-01 06:23:30 [autobid.de] INFO: Scrape type for 3361697: 1
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361698%27 HTTP/1.1" 200 None
2026-04-01 06:23:30 [autobid.de] INFO: Scrape type for 3361698: 1
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366524%27 HTTP/1.1" 200 None
2026-04-01 06:23:30 [autobid.de] INFO: Scrape type for 3366524: 1
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366525%27 HTTP/1.1" 200 None
2026-04-01 06:23:30 [autobid.de] INFO: Scrape type for 3366525: 1
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366531%27 HTTP/1.1" 200 None
2026-04-01 06:23:30 [autobid.de] INFO: Scrape type for 3366531: 1
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360135%27 HTTP/1.1" 200 None
2026-04-01 06:23:30 [autobid.de] INFO: Scrape type for 3360135: 1
2026-04-01 06:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361695%27 HTTP/1.1" 200 None
2026-04-01 06:23:31 [autobid.de] INFO: Scrape type for 3361695: 1
2026-04-01 06:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361692%27 HTTP/1.1" 200 None
2026-04-01 06:23:31 [autobid.de] INFO: Scrape type for 3361692: 1
2026-04-01 06:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360141%27 HTTP/1.1" 200 None
2026-04-01 06:23:31 [autobid.de] INFO: Scrape type for 3360141: 1
2026-04-01 06:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361700%27 HTTP/1.1" 200 None
2026-04-01 06:23:31 [autobid.de] INFO: Scrape type for 3361700: 1
2026-04-01 06:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360140%27 HTTP/1.1" 200 None
2026-04-01 06:23:31 [autobid.de] INFO: Scrape type for 3360140: 1
2026-04-01 06:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366554%27 HTTP/1.1" 200 None
2026-04-01 06:23:31 [autobid.de] INFO: Scrape type for 3366554: 1
2026-04-01 06:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366555%27 HTTP/1.1" 200 None
2026-04-01 06:23:31 [autobid.de] INFO: Scrape type for 3366555: 1
2026-04-01 06:23:42 [scrapy.extensions.logstats] INFO: Crawled 115 pages (at 6 pages/min), scraped 3 items (at 0 items/min)
2026-04-01 06:23:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 220MiB
2026-04-01 06:24:42 [scrapy.extensions.logstats] INFO: Crawled 115 pages (at 0 pages/min), scraped 3 items (at 0 items/min)
2026-04-01 06:24:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 220MiB
2026-04-01 06:25:42 [scrapy.extensions.logstats] INFO: Crawled 115 pages (at 0 pages/min), scraped 3 items (at 0 items/min)
2026-04-01 06:25:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 220MiB
2026-04-01 06:26:20 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 07299513-4d2c-5e69-bfc3-ca4ca75c839f with identifier: 3361694
2026-04-01 06:26:20 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 06:26:20 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=grey+Metallic HTTP/1.1" 200 None
2026-04-01 06:26:20 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 06:26:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 06:26:20 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 07299513-4d2c-5e69-bfc3-ca4ca75c839f with identifier: 3361694
2026-04-01 06:26:20 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: 07299513-4d2c-5e69-bfc3-ca4ca75c839f with identifier: 3361694 to the API
2026-04-01 06:26:20 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 06:26:20 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 06:26:21 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33617/3361694/93586431_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 81381',
'body_type': 'All-terrain vehicle',
'category': 'All-terrain vehicle',
'color': 'grey Metallic',
'currency': 'EUR',
'emission_standard': 'EURO 6',
'engine': '1.4L',
'engine_horse_power': 150,
'expiration_date': '2026-04-01T09:00:00Z',
'fuel_type': 'Petrol',
'id': '07299513-4d2c-5e69-bfc3-ca4ca75c839f',
'is_damaged': False,
'is_operable': True,
'is_vat_deductible': False,
'link_to_web_offer': 'https://autobid.de/en/item/vw-tiguan-1-4-tsi-bmt-dsg-sound-3361694',
'listing': 'auction',
'location': 'Magdeburg',
'make': 'VW',
'model': 'Tiguan',
'odometer': 137000,
'odometer_reading_unit': 'km',
'origin_country_code': 'DE',
'original_photo_count': 5,
'photo_urls': ['https://cdn.autobid.de/data/cars/33617/3361694/93586431_hd.jpg',
'https://cdn.autobid.de/data/cars/33617/3361694/93586432_hd.jpg',
'https://cdn.autobid.de/data/cars/33617/3361694/93586433_hd.jpg',
'https://cdn.autobid.de/data/cars/33617/3361694/93586434_hd.jpg',
'https://cdn.autobid.de/data/cars/33617/3361694/93586435_hd.jpg'],
'photos': [],
'price': 10900,
'price_includes_vat': False,
'registration_date': '2017-05-01',
'seats_number': 5,
'seller_name': 'Subsidiary/authorised dealer of the brand(s): Audi, VW',
'title': 'VW Tiguan 1,4 TSI BMT DSG "Sound"',
'transmission': 'automatic',
'vin': None,
'year': 2017}
2026-04-01 06:26:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361694%27 HTTP/1.1" 200 None
2026-04-01 06:26:21 [autobid.de] INFO: Saving data for 3361694: {'created_time': 1775024781.041201, 'last_price_update_time': 1775024781.041213, 'auction_closing_time': 1775034000.0}
2026-04-01 06:26:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3361694') HTTP/1.1" 204 0
2026-04-01 06:26:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 07299513-4d2c-5e69-bfc3-ca4ca75c839f due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33617/3361694/93586431_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:26:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 70745f8f-d834-5336-91d3-7e3f98559def due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33652/3365130/93563371_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:26:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=18> (referer: None)
2026-04-01 06:26:22 [autobid.de] INFO: Page 18/146: found 20 items
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364619%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [autobid.de] INFO: Scrape type for 3364619: 1
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347977%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [autobid.de] INFO: Scrape type for 3347977: 1
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364618%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [autobid.de] INFO: Scrape type for 3364618: 1
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352453%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [autobid.de] INFO: Scrape type for 3352453: 1
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364606%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [autobid.de] INFO: Scrape type for 3364606: 1
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354809%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [autobid.de] INFO: Scrape type for 3354809: 1
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364629%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [autobid.de] INFO: Scrape type for 3364629: 1
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355045%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [autobid.de] INFO: Scrape type for 3355045: 1
2026-04-01 06:26:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/citroen-berlingo-kasten-1-5-blue-hdi-club-3366555/details> (referer: None)
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364605%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [autobid.de] INFO: Scrape type for 3364605: 1
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359084%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [autobid.de] INFO: Scrape type for 3359084: 1
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364622%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [autobid.de] INFO: Scrape type for 3364622: 1
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359094%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [autobid.de] INFO: Scrape type for 3359094: 1
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364615%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [autobid.de] INFO: Scrape type for 3364615: 1
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359096%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [autobid.de] INFO: Scrape type for 3359096: 1
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364613%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [autobid.de] INFO: Scrape type for 3364613: 1
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359097%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [autobid.de] INFO: Scrape type for 3359097: 1
2026-04-01 06:26:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: df6fa2cb-ebc8-535a-a743-c9136bac3187 with identifier: 3366555
2026-04-01 06:26:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item df6fa2cb-ebc8-535a-a743-c9136bac3187 with scrape type 1
2026-04-01 06:26:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item df6fa2cb-ebc8-535a-a743-c9136bac3187
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33666%252F3366555%252F93640966_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33666/3366555/93640966_hd.jpg not downloaded yet
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364617%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [autobid.de] INFO: Scrape type for 3364617: 1
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33666%252F3366555%252F93640947_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33666/3366555/93640947_hd.jpg not downloaded yet
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363729%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [autobid.de] INFO: Scrape type for 3363729: 1
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33666%252F3366555%252F93640958_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33666/3366555/93640958_hd.jpg not downloaded yet
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364621%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [autobid.de] INFO: Scrape type for 3364621: 1
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33666%252F3366555%252F93640951_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33666/3366555/93640951_hd.jpg not downloaded yet
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363730%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [autobid.de] INFO: Scrape type for 3363730: 1
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33666%252F3366555%252F93640952_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33666/3366555/93640952_hd.jpg not downloaded yet
2026-04-01 06:26:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/hyundai-i30-1-6-t-gdi-3366554/details> (referer: None)
2026-04-01 06:26:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 1caede32-b60b-5a17-957d-7c034057e8f5 with identifier: 3366554
2026-04-01 06:26:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 1caede32-b60b-5a17-957d-7c034057e8f5 with scrape type 1
2026-04-01 06:26:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 1caede32-b60b-5a17-957d-7c034057e8f5
2026-04-01 06:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33666%252F3366554%252F93640918_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33666/3366554/93640918_hd.jpg not downloaded yet
2026-04-01 06:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33666%252F3366554%252F93640921_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33666/3366554/93640921_hd.jpg not downloaded yet
2026-04-01 06:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33666%252F3366554%252F93640916_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33666/3366554/93640916_hd.jpg not downloaded yet
2026-04-01 06:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33666%252F3366554%252F93640920_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33666/3366554/93640920_hd.jpg not downloaded yet
2026-04-01 06:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33666%252F3366554%252F93640931_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33666/3366554/93640931_hd.jpg not downloaded yet
2026-04-01 06:26:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 70745f8f-d834-5336-91d3-7e3f98559def due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33652/3365130/93563382_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:26:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-polo-1-4-mpi-comfortline-3355045/details> (referer: None)
2026-04-01 06:26:24 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: c61183a1-512a-5144-a0dd-e3f29a627b7e with identifier: 3355045
2026-04-01 06:26:24 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item c61183a1-512a-5144-a0dd-e3f29a627b7e with scrape type 1
2026-04-01 06:26:24 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item c61183a1-512a-5144-a0dd-e3f29a627b7e
2026-04-01 06:26:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33551%252F3355045%252F93459113_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33551/3355045/93459113_hd.jpg not downloaded yet
2026-04-01 06:26:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33551%252F3355045%252F93459114_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33551/3355045/93459114_hd.jpg not downloaded yet
2026-04-01 06:26:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33551%252F3355045%252F93459115_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33551/3355045/93459115_hd.jpg not downloaded yet
2026-04-01 06:26:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33551%252F3355045%252F93459116_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33551/3355045/93459116_hd.jpg not downloaded yet
2026-04-01 06:26:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33551%252F3355045%252F93459117_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33551/3355045/93459117_hd.jpg not downloaded yet
2026-04-01 06:26:25 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 70745f8f-d834-5336-91d3-7e3f98559def with identifier: 3365130
2026-04-01 06:26:25 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 06:26:25 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=black+Pearl+effect HTTP/1.1" 200 None
2026-04-01 06:26:25 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 06:26:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 06:26:25 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 70745f8f-d834-5336-91d3-7e3f98559def with identifier: 3365130
2026-04-01 06:26:25 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: 70745f8f-d834-5336-91d3-7e3f98559def with identifier: 3365130 to the API
2026-04-01 06:26:25 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 06:26:26 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 06:26:26 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33652/3365130/93563389_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 81381',
'body_type': 'Estate',
'category': 'Estate',
'color': 'black Pearl effect',
'currency': 'EUR',
'emission_standard': 'EURO 5',
'engine': '1.4L',
'engine_horse_power': 122,
'expiration_date': '2026-04-01T09:00:00Z',
'fuel_type': 'Petrol',
'id': '70745f8f-d834-5336-91d3-7e3f98559def',
'is_damaged': False,
'is_operable': True,
'is_vat_deductible': False,
'link_to_web_offer': 'https://autobid.de/en/item/vw-passat-variant-1-4-tsi-bmt-comfortline-3365130',
'listing': 'auction',
'location': 'Plauen',
'make': 'VW',
'model': 'Passat Variant',
'odometer': 141500,
'odometer_reading_unit': 'km',
'origin_country_code': 'DE',
'original_photo_count': 5,
'photo_urls': ['https://cdn.autobid.de/data/cars/33652/3365130/93563389_hd.jpg',
'https://cdn.autobid.de/data/cars/33652/3365130/93563382_hd.jpg',
'https://cdn.autobid.de/data/cars/33652/3365130/93563371_hd.jpg',
'https://cdn.autobid.de/data/cars/33652/3365130/93563374_hd.jpg',
'https://cdn.autobid.de/data/cars/33652/3365130/93563384_hd.jpg'],
'photos': [],
'price': 4500,
'price_includes_vat': False,
'registration_date': '2013-05-01',
'seats_number': 5,
'seller_name': 'Subsidiary/authorised dealer of the brand(s): Renault, Land '
'Rover, Dacia, Seat, Jaguar',
'title': 'VW Passat Variant 1,4 TSI BMT "Comfortline"',
'transmission': 'manual',
'vin': None,
'year': 2013}
2026-04-01 06:26:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365130%27 HTTP/1.1" 200 None
2026-04-01 06:26:26 [autobid.de] INFO: Saving data for 3365130: {'created_time': 1775024786.254554, 'last_price_update_time': 1775024786.254566, 'auction_closing_time': 1775034000.0}
2026-04-01 06:26:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3365130') HTTP/1.1" 204 0
2026-04-01 06:26:26 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 70745f8f-d834-5336-91d3-7e3f98559def due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33652/3365130/93563389_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:26:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 5c939d81-707d-5404-b3e3-79590a8cad81 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33638/3363701/93533516_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:26:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=19> (referer: None)
2026-04-01 06:26:27 [autobid.de] INFO: Page 19/146: found 20 items
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364637%27 HTTP/1.1" 200 None
2026-04-01 06:26:27 [autobid.de] INFO: Scrape type for 3364637: 1
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363731%27 HTTP/1.1" 200 None
2026-04-01 06:26:27 [autobid.de] INFO: Scrape type for 3363731: 1
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364642%27 HTTP/1.1" 200 None
2026-04-01 06:26:27 [autobid.de] INFO: Scrape type for 3364642: 1
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365707%27 HTTP/1.1" 200 None
2026-04-01 06:26:27 [autobid.de] INFO: Scrape type for 3365707: 1
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364612%27 HTTP/1.1" 200 None
2026-04-01 06:26:27 [autobid.de] INFO: Scrape type for 3364612: 1
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365708%27 HTTP/1.1" 200 None
2026-04-01 06:26:27 [autobid.de] INFO: Scrape type for 3365708: 1
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364688%27 HTTP/1.1" 200 None
2026-04-01 06:26:27 [autobid.de] INFO: Scrape type for 3364688: 1
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365709%27 HTTP/1.1" 200 None
2026-04-01 06:26:27 [autobid.de] INFO: Scrape type for 3365709: 1
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364739%27 HTTP/1.1" 200 None
2026-04-01 06:26:27 [autobid.de] INFO: Scrape type for 3364739: 1
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365710%27 HTTP/1.1" 200 None
2026-04-01 06:26:27 [autobid.de] INFO: Scrape type for 3365710: 1
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364738%27 HTTP/1.1" 200 None
2026-04-01 06:26:27 [autobid.de] INFO: Scrape type for 3364738: 1
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365711%27 HTTP/1.1" 200 None
2026-04-01 06:26:27 [autobid.de] INFO: Scrape type for 3365711: 1
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364711%27 HTTP/1.1" 200 None
2026-04-01 06:26:27 [autobid.de] INFO: Scrape type for 3364711: 1
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361470%27 HTTP/1.1" 200 None
2026-04-01 06:26:27 [autobid.de] INFO: Scrape type for 3361470: 1
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364745%27 HTTP/1.1" 200 None
2026-04-01 06:26:27 [autobid.de] INFO: Scrape type for 3364745: 1
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361744%27 HTTP/1.1" 200 None
2026-04-01 06:26:27 [autobid.de] INFO: Scrape type for 3361744: 1
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364743%27 HTTP/1.1" 200 None
2026-04-01 06:26:27 [autobid.de] INFO: Scrape type for 3364743: 1
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362724%27 HTTP/1.1" 200 None
2026-04-01 06:26:27 [autobid.de] INFO: Scrape type for 3362724: 1
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364744%27 HTTP/1.1" 200 None
2026-04-01 06:26:27 [autobid.de] INFO: Scrape type for 3364744: 1
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359072%27 HTTP/1.1" 200 None
2026-04-01 06:26:27 [autobid.de] INFO: Scrape type for 3359072: 1
2026-04-01 06:26:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 5c939d81-707d-5404-b3e3-79590a8cad81 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33638/3363701/93533511_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:26:29 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 5c939d81-707d-5404-b3e3-79590a8cad81 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33638/3363701/93533527_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:26:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ford-focus-1-0-ecoboost-titanium-3359072/details> (referer: None)
2026-04-01 06:26:30 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 20144b9b-6632-5479-aca8-14b2667aa571 with identifier: 3359072
2026-04-01 06:26:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 20144b9b-6632-5479-aca8-14b2667aa571 with scrape type 1
2026-04-01 06:26:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 20144b9b-6632-5479-aca8-14b2667aa571
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359072%252F93528177_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359072/93528177_hd.jpg not downloaded yet
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359072%252F93528178_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359072/93528178_hd.jpg not downloaded yet
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359072%252F93528179_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359072/93528179_hd.jpg not downloaded yet
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359072%252F93528180_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359072/93528180_hd.jpg not downloaded yet
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359072%252F93528181_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359072/93528181_hd.jpg not downloaded yet
2026-04-01 06:26:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=20> (referer: None)
2026-04-01 06:26:30 [autobid.de] INFO: Page 20/146: found 20 items
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364741%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [autobid.de] INFO: Scrape type for 3364741: 1
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359073%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [autobid.de] INFO: Scrape type for 3359073: 1
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364742%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [autobid.de] INFO: Scrape type for 3364742: 1
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359074%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [autobid.de] INFO: Scrape type for 3359074: 1
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364740%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [autobid.de] INFO: Scrape type for 3364740: 1
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359889%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [autobid.de] INFO: Scrape type for 3359889: 1
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364761%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [autobid.de] INFO: Scrape type for 3364761: 1
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360051%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [autobid.de] INFO: Scrape type for 3360051: 1
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364710%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [autobid.de] INFO: Scrape type for 3364710: 1
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360519%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [autobid.de] INFO: Scrape type for 3360519: 1
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364626%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [autobid.de] INFO: Scrape type for 3364626: 1
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363763%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [autobid.de] INFO: Scrape type for 3363763: 1
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364640%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [autobid.de] INFO: Scrape type for 3364640: 1
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361813%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [autobid.de] INFO: Scrape type for 3361813: 1
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364759%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [autobid.de] INFO: Scrape type for 3364759: 1
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361743%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [autobid.de] INFO: Scrape type for 3361743: 1
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364762%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [autobid.de] INFO: Scrape type for 3364762: 1
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354810%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [autobid.de] INFO: Scrape type for 3354810: 1
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364763%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [autobid.de] INFO: Scrape type for 3364763: 1
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354808%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [autobid.de] INFO: Scrape type for 3354808: 1
2026-04-01 06:26:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-tiguan-allspace-2-0-tdi-4motion-dsg-r-line-3364744/details> (referer: None)
2026-04-01 06:26:30 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 50fc0989-fe2d-5586-80c6-7e2254be6419 with identifier: 3364744
2026-04-01 06:26:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 50fc0989-fe2d-5586-80c6-7e2254be6419 with scrape type 1
2026-04-01 06:26:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 50fc0989-fe2d-5586-80c6-7e2254be6419
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33648%252F3364744%252F93546590_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33648/3364744/93546590_hd.jpg not downloaded yet
2026-04-01 06:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33648%252F3364744%252F93546584_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33648/3364744/93546584_hd.jpg not downloaded yet
2026-04-01 06:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33648%252F3364744%252F93546576_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33648/3364744/93546576_hd.jpg not downloaded yet
2026-04-01 06:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33648%252F3364744%252F93546574_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33648/3364744/93546574_hd.jpg not downloaded yet
2026-04-01 06:26:31 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 5c939d81-707d-5404-b3e3-79590a8cad81 with identifier: 3363701
2026-04-01 06:26:31 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 06:26:31 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=Hybrid HTTP/1.1" 200 None
2026-04-01 06:26:31 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 06:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 06:26:31 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 5c939d81-707d-5404-b3e3-79590a8cad81 with identifier: 3363701
2026-04-01 06:26:31 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: 5c939d81-707d-5404-b3e3-79590a8cad81 with identifier: 3363701 to the API
2026-04-01 06:26:31 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 06:26:31 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 06:26:31 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33638/3363701/93533503_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 81381',
'body_type': 'Saloon',
'category': 'Saloon',
'color': 'silver Metallic',
'currency': 'EUR',
'emission_standard': 'EURO 6',
'engine': '3.5L',
'engine_horse_power': 299,
'expiration_date': '2026-04-01T09:00:00Z',
'fuel_type': 'Hybrid',
'id': '5c939d81-707d-5404-b3e3-79590a8cad81',
'is_damaged': False,
'is_operable': True,
'is_vat_included': True,
'link_to_web_offer': 'https://autobid.de/en/item/lexus-ls500h-awd-3363701',
'listing': 'auction',
'location': 'Landsberg OT Peißen',
'make': 'Lexus',
'model': 'LS500h AWD',
'odometer': 44100,
'odometer_reading_unit': 'km',
'origin_country_code': 'DE',
'original_photo_count': 5,
'photo_urls': ['https://cdn.autobid.de/data/cars/33638/3363701/93533503_hd.jpg',
'https://cdn.autobid.de/data/cars/33638/3363701/93533527_hd.jpg',
'https://cdn.autobid.de/data/cars/33638/3363701/93533511_hd.jpg',
'https://cdn.autobid.de/data/cars/33638/3363701/93533516_hd.jpg',
'https://cdn.autobid.de/data/cars/33638/3363701/93533526_hd.jpg'],
'photos': [],
'price': 48200,
'price_includes_vat': True,
'registration_date': '2023-06-01',
'seats_number': 5,
'seller_name': None,
'title': 'Lexus LS500h AWD',
'transmission': 'automatic',
'vin': None,
'year': 2023}
2026-04-01 06:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363701%27 HTTP/1.1" 200 None
2026-04-01 06:26:31 [autobid.de] INFO: Saving data for 3363701: {'created_time': 1775024791.784449, 'last_price_update_time': 1775024791.784462, 'auction_closing_time': 1775034000.0}
2026-04-01 06:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3363701') HTTP/1.1" 204 0
2026-04-01 06:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33648%252F3364744%252F93546605_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33648/3364744/93546605_hd.jpg not downloaded yet
2026-04-01 06:26:31 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 5c939d81-707d-5404-b3e3-79590a8cad81 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33638/3363701/93533503_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:26:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/skoda-kamiq-1-5-tsi-act-dsg-style-3354808/details> (referer: None)
2026-04-01 06:26:33 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: b546be6e-ebde-53c8-93e6-bd5809aeb371 with identifier: 3354808
2026-04-01 06:26:33 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b546be6e-ebde-53c8-93e6-bd5809aeb371 with scrape type 1
2026-04-01 06:26:33 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item b546be6e-ebde-53c8-93e6-bd5809aeb371
2026-04-01 06:26:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33549%252F3354808%252F93545906_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33549/3354808/93545906_hd.jpg not downloaded yet
2026-04-01 06:26:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33549%252F3354808%252F93545909_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33549/3354808/93545909_hd.jpg not downloaded yet
2026-04-01 06:26:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33549%252F3354808%252F93545925_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33549/3354808/93545925_hd.jpg not downloaded yet
2026-04-01 06:26:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33549%252F3354808%252F93545922_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33549/3354808/93545922_hd.jpg not downloaded yet
2026-04-01 06:26:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33549%252F3354808%252F93545917_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:26:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33549/3354808/93545917_hd.jpg not downloaded yet
2026-04-01 06:26:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=21> (referer: None)
2026-04-01 06:26:34 [autobid.de] INFO: Page 21/146: found 20 items
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365058%27 HTTP/1.1" 200 None
2026-04-01 06:26:34 [autobid.de] INFO: Scrape type for 3365058: 1
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352452%27 HTTP/1.1" 200 None
2026-04-01 06:26:34 [autobid.de] INFO: Scrape type for 3352452: 1
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365057%27 HTTP/1.1" 200 None
2026-04-01 06:26:34 [autobid.de] INFO: Scrape type for 3365057: 1
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273342539%27 HTTP/1.1" 200 None
2026-04-01 06:26:34 [autobid.de] INFO: Scrape type for 3342539: 1
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365059%27 HTTP/1.1" 200 None
2026-04-01 06:26:34 [autobid.de] INFO: Scrape type for 3365059: 1
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273325880%27 HTTP/1.1" 200 None
2026-04-01 06:26:34 [autobid.de] INFO: Scrape type for 3325880: 1
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365056%27 HTTP/1.1" 200 None
2026-04-01 06:26:34 [autobid.de] INFO: Scrape type for 3365056: 1
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366909%27 HTTP/1.1" 200 None
2026-04-01 06:26:34 [autobid.de] INFO: Scrape type for 3366909: 1
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365024%27 HTTP/1.1" 200 None
2026-04-01 06:26:34 [autobid.de] INFO: Scrape type for 3365024: 1
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366954%27 HTTP/1.1" 200 None
2026-04-01 06:26:34 [autobid.de] INFO: Scrape type for 3366954: 1
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365060%27 HTTP/1.1" 200 None
2026-04-01 06:26:34 [autobid.de] INFO: Scrape type for 3365060: 1
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366955%27 HTTP/1.1" 200 None
2026-04-01 06:26:34 [autobid.de] INFO: Scrape type for 3366955: 1
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365061%27 HTTP/1.1" 200 None
2026-04-01 06:26:34 [autobid.de] INFO: Scrape type for 3365061: 1
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366956%27 HTTP/1.1" 200 None
2026-04-01 06:26:34 [autobid.de] INFO: Scrape type for 3366956: 1
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365062%27 HTTP/1.1" 200 None
2026-04-01 06:26:34 [autobid.de] INFO: Scrape type for 3365062: 1
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366957%27 HTTP/1.1" 200 None
2026-04-01 06:26:34 [autobid.de] INFO: Scrape type for 3366957: 1
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365149%27 HTTP/1.1" 200 None
2026-04-01 06:26:34 [autobid.de] INFO: Scrape type for 3365149: 1
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366958%27 HTTP/1.1" 200 None
2026-04-01 06:26:34 [autobid.de] INFO: Scrape type for 3366958: 1
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365150%27 HTTP/1.1" 200 None
2026-04-01 06:26:34 [autobid.de] INFO: Scrape type for 3365150: 1
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:26:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366959%27 HTTP/1.1" 200 None
2026-04-01 06:26:34 [autobid.de] INFO: Scrape type for 3366959: 1
2026-04-01 06:26:42 [scrapy.extensions.logstats] INFO: Crawled 125 pages (at 10 pages/min), scraped 6 items (at 3 items/min)
2026-04-01 06:26:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 220MiB
2026-04-01 06:27:42 [scrapy.extensions.logstats] INFO: Crawled 125 pages (at 0 pages/min), scraped 6 items (at 0 items/min)
2026-04-01 06:27:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 220MiB
2026-04-01 06:28:42 [scrapy.extensions.logstats] INFO: Crawled 125 pages (at 0 pages/min), scraped 6 items (at 0 items/min)
2026-04-01 06:28:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 220MiB
2026-04-01 06:29:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 741026e9-3931-53d3-8e0a-bc0ab3c159d4 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33617/3361693/93582834_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:29:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 881d9bea-9951-513f-8a14-27e5bc036d7a due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33576/3357584/93583202_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:29:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-a5-sportback-40-tfsi-s-tronic-3366959/details> (referer: None)
2026-04-01 06:29:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 4770e7c0-e5ef-5288-95bc-08bcd0b0bbc6 with identifier: 3366959
2026-04-01 06:29:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 4770e7c0-e5ef-5288-95bc-08bcd0b0bbc6 with scrape type 1
2026-04-01 06:29:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 4770e7c0-e5ef-5288-95bc-08bcd0b0bbc6
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33670%252F3366959%252F93652356_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33670/3366959/93652356_hd.jpg not downloaded yet
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33670%252F3366959%252F93652365_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33670/3366959/93652365_hd.jpg not downloaded yet
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33670%252F3366959%252F93652359_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33670/3366959/93652359_hd.jpg not downloaded yet
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33670%252F3366959%252F93652371_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33670/3366959/93652371_hd.jpg not downloaded yet
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33670%252F3366959%252F93652354_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33670/3366959/93652354_hd.jpg not downloaded yet
2026-04-01 06:29:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=22> (referer: None)
2026-04-01 06:29:22 [autobid.de] INFO: Page 22/146: found 20 items
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366961%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [autobid.de] INFO: Scrape type for 3366961: 1
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365253%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [autobid.de] INFO: Scrape type for 3365253: 1
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366962%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [autobid.de] INFO: Scrape type for 3366962: 1
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365259%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [autobid.de] INFO: Scrape type for 3365259: 1
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366963%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [autobid.de] INFO: Scrape type for 3366963: 1
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365261%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [autobid.de] INFO: Scrape type for 3365261: 1
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366964%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [autobid.de] INFO: Scrape type for 3366964: 1
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365545%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [autobid.de] INFO: Scrape type for 3365545: 1
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366965%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [autobid.de] INFO: Scrape type for 3366965: 1
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365546%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [autobid.de] INFO: Scrape type for 3365546: 1
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366966%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [autobid.de] INFO: Scrape type for 3366966: 1
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365549%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [autobid.de] INFO: Scrape type for 3365549: 1
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366967%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [autobid.de] INFO: Scrape type for 3366967: 1
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365552%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [autobid.de] INFO: Scrape type for 3365552: 1
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366968%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [autobid.de] INFO: Scrape type for 3366968: 1
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365551%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [autobid.de] INFO: Scrape type for 3365551: 1
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366969%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [autobid.de] INFO: Scrape type for 3366969: 1
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366375%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [autobid.de] INFO: Scrape type for 3366375: 1
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366970%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [autobid.de] INFO: Scrape type for 3366970: 1
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366377%27 HTTP/1.1" 200 None
2026-04-01 06:29:22 [autobid.de] INFO: Scrape type for 3366377: 1
2026-04-01 06:29:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mercedes-benz-v-250-lang-cdi-4matic-gtronic-avantgarde-edition-3365150/details> (referer: None)
2026-04-01 06:29:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 976d2125-f0b4-5b61-a20f-a395fafc9110 with identifier: 3365150
2026-04-01 06:29:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 976d2125-f0b4-5b61-a20f-a395fafc9110 with scrape type 1
2026-04-01 06:29:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 976d2125-f0b4-5b61-a20f-a395fafc9110
2026-04-01 06:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33652%252F3365150%252F93565087_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33652/3365150/93565087_hd.jpg not downloaded yet
2026-04-01 06:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33652%252F3365150%252F93565089_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33652/3365150/93565089_hd.jpg not downloaded yet
2026-04-01 06:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33652%252F3365150%252F93565105_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33652/3365150/93565105_hd.jpg not downloaded yet
2026-04-01 06:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33652%252F3365150%252F93565104_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33652/3365150/93565104_hd.jpg not downloaded yet
2026-04-01 06:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33652%252F3365150%252F93565088_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33652/3365150/93565088_hd.jpg not downloaded yet
2026-04-01 06:29:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 881d9bea-9951-513f-8a14-27e5bc036d7a due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33576/3357584/93583204_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:29:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 881d9bea-9951-513f-8a14-27e5bc036d7a due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33576/3357584/93583203_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:29:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=23> (referer: None)
2026-04-01 06:29:26 [autobid.de] INFO: Page 23/146: found 20 items
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366971%27 HTTP/1.1" 200 None
2026-04-01 06:29:26 [autobid.de] INFO: Scrape type for 3366971: 1
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366374%27 HTTP/1.1" 200 None
2026-04-01 06:29:26 [autobid.de] INFO: Scrape type for 3366374: 1
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366972%27 HTTP/1.1" 200 None
2026-04-01 06:29:26 [autobid.de] INFO: Scrape type for 3366972: 1
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365087%27 HTTP/1.1" 200 None
2026-04-01 06:29:26 [autobid.de] INFO: Scrape type for 3365087: 1
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366973%27 HTTP/1.1" 200 None
2026-04-01 06:29:26 [autobid.de] INFO: Scrape type for 3366973: 1
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364766%27 HTTP/1.1" 200 None
2026-04-01 06:29:26 [autobid.de] INFO: Scrape type for 3364766: 1
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354811%27 HTTP/1.1" 200 None
2026-04-01 06:29:26 [autobid.de] INFO: Scrape type for 3354811: 1
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365201%27 HTTP/1.1" 200 None
2026-04-01 06:29:26 [autobid.de] INFO: Scrape type for 3365201: 1
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365127%27 HTTP/1.1" 200 None
2026-04-01 06:29:26 [autobid.de] INFO: Scrape type for 3365127: 1
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365115%27 HTTP/1.1" 200 None
2026-04-01 06:29:26 [autobid.de] INFO: Scrape type for 3365115: 1
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365248%27 HTTP/1.1" 200 None
2026-04-01 06:29:26 [autobid.de] INFO: Scrape type for 3365248: 1
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365250%27 HTTP/1.1" 200 None
2026-04-01 06:29:26 [autobid.de] INFO: Scrape type for 3365250: 1
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365254%27 HTTP/1.1" 200 None
2026-04-01 06:29:26 [autobid.de] INFO: Scrape type for 3365254: 1
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365256%27 HTTP/1.1" 200 None
2026-04-01 06:29:26 [autobid.de] INFO: Scrape type for 3365256: 1
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365249%27 HTTP/1.1" 200 None
2026-04-01 06:29:26 [autobid.de] INFO: Scrape type for 3365249: 1
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365246%27 HTTP/1.1" 200 None
2026-04-01 06:29:26 [autobid.de] INFO: Scrape type for 3365246: 1
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365258%27 HTTP/1.1" 200 None
2026-04-01 06:29:26 [autobid.de] INFO: Scrape type for 3365258: 1
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365260%27 HTTP/1.1" 200 None
2026-04-01 06:29:26 [autobid.de] INFO: Scrape type for 3365260: 1
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365262%27 HTTP/1.1" 200 None
2026-04-01 06:29:26 [autobid.de] INFO: Scrape type for 3365262: 1
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365263%27 HTTP/1.1" 200 None
2026-04-01 06:29:26 [autobid.de] INFO: Scrape type for 3365263: 1
2026-04-01 06:29:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 54fe1c0c-3e8f-5551-ba9f-9b5bd3df09e8 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33656/3365567/93586069_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:29:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 54fe1c0c-3e8f-5551-ba9f-9b5bd3df09e8 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33656/3365567/93586050_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:29:28 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-a3-30-tdi-advanced-3365263/details> (referer: None)
2026-04-01 06:29:28 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 70ce78c2-b4ae-5b12-8981-d91ab9a9e6c8 with identifier: 3365263
2026-04-01 06:29:28 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 70ce78c2-b4ae-5b12-8981-d91ab9a9e6c8 with scrape type 1
2026-04-01 06:29:28 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 70ce78c2-b4ae-5b12-8981-d91ab9a9e6c8
2026-04-01 06:29:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33653%252F3365263%252F93580287_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33653/3365263/93580287_hd.jpg not downloaded yet
2026-04-01 06:29:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33653%252F3365263%252F93580288_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33653/3365263/93580288_hd.jpg not downloaded yet
2026-04-01 06:29:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33653%252F3365263%252F93580284_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33653/3365263/93580284_hd.jpg not downloaded yet
2026-04-01 06:29:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33653%252F3365263%252F93580293_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33653/3365263/93580293_hd.jpg not downloaded yet
2026-04-01 06:29:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33653%252F3365263%252F93580303_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33653/3365263/93580303_hd.jpg not downloaded yet
2026-04-01 06:29:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mercedes-benz-b-220-4matic-dct-progressive-3365262/details> (referer: None)
2026-04-01 06:29:29 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 6b57c858-513a-5f76-a56e-7b962433d48c with identifier: 3365262
2026-04-01 06:29:29 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6b57c858-513a-5f76-a56e-7b962433d48c with scrape type 1
2026-04-01 06:29:29 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 6b57c858-513a-5f76-a56e-7b962433d48c
2026-04-01 06:29:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33653%252F3365262%252F93580256_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33653/3365262/93580256_hd.jpg not downloaded yet
2026-04-01 06:29:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33653%252F3365262%252F93580261_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33653/3365262/93580261_hd.jpg not downloaded yet
2026-04-01 06:29:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33653%252F3365262%252F93580272_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33653/3365262/93580272_hd.jpg not downloaded yet
2026-04-01 06:29:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33653%252F3365262%252F93580255_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33653/3365262/93580255_hd.jpg not downloaded yet
2026-04-01 06:29:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33653%252F3365262%252F93580260_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33653/3365262/93580260_hd.jpg not downloaded yet
2026-04-01 06:29:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 54fe1c0c-3e8f-5551-ba9f-9b5bd3df09e8 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33656/3365567/93586065_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:29:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=24> (referer: None)
2026-04-01 06:29:30 [autobid.de] INFO: Page 24/146: found 20 items
2026-04-01 06:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365537%27 HTTP/1.1" 200 None
2026-04-01 06:29:30 [autobid.de] INFO: Scrape type for 3365537: 1
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365538%27 HTTP/1.1" 200 None
2026-04-01 06:29:31 [autobid.de] INFO: Scrape type for 3365538: 1
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365257%27 HTTP/1.1" 200 None
2026-04-01 06:29:31 [autobid.de] INFO: Scrape type for 3365257: 1
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364767%27 HTTP/1.1" 200 None
2026-04-01 06:29:31 [autobid.de] INFO: Scrape type for 3364767: 1
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365547%27 HTTP/1.1" 200 None
2026-04-01 06:29:31 [autobid.de] INFO: Scrape type for 3365547: 1
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365548%27 HTTP/1.1" 200 None
2026-04-01 06:29:31 [autobid.de] INFO: Scrape type for 3365548: 1
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365550%27 HTTP/1.1" 200 None
2026-04-01 06:29:31 [autobid.de] INFO: Scrape type for 3365550: 1
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365553%27 HTTP/1.1" 200 None
2026-04-01 06:29:31 [autobid.de] INFO: Scrape type for 3365553: 1
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365554%27 HTTP/1.1" 200 None
2026-04-01 06:29:31 [autobid.de] INFO: Scrape type for 3365554: 1
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365555%27 HTTP/1.1" 200 None
2026-04-01 06:29:31 [autobid.de] INFO: Scrape type for 3365555: 1
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365556%27 HTTP/1.1" 200 None
2026-04-01 06:29:31 [autobid.de] INFO: Scrape type for 3365556: 1
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365558%27 HTTP/1.1" 200 None
2026-04-01 06:29:31 [autobid.de] INFO: Scrape type for 3365558: 1
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366373%27 HTTP/1.1" 200 None
2026-04-01 06:29:31 [autobid.de] INFO: Scrape type for 3366373: 1
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365559%27 HTTP/1.1" 200 None
2026-04-01 06:29:31 [autobid.de] INFO: Scrape type for 3365559: 1
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365199%27 HTTP/1.1" 200 None
2026-04-01 06:29:31 [autobid.de] INFO: Scrape type for 3365199: 1
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365088%27 HTTP/1.1" 200 None
2026-04-01 06:29:31 [autobid.de] INFO: Scrape type for 3365088: 1
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365247%27 HTTP/1.1" 200 None
2026-04-01 06:29:31 [autobid.de] INFO: Scrape type for 3365247: 1
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365264%27 HTTP/1.1" 200 None
2026-04-01 06:29:31 [autobid.de] INFO: Scrape type for 3365264: 1
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365126%27 HTTP/1.1" 200 None
2026-04-01 06:29:31 [autobid.de] INFO: Scrape type for 3365126: 1
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364644%27 HTTP/1.1" 200 None
2026-04-01 06:29:31 [autobid.de] INFO: Scrape type for 3364644: 1
2026-04-01 06:29:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 54fe1c0c-3e8f-5551-ba9f-9b5bd3df09e8 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33656/3365567/93586063_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:29:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=25> (referer: None)
2026-04-01 06:29:34 [autobid.de] INFO: Page 25/146: found 20 items
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364736%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [autobid.de] INFO: Scrape type for 3364736: 1
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364746%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [autobid.de] INFO: Scrape type for 3364746: 1
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364737%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [autobid.de] INFO: Scrape type for 3364737: 1
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364735%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [autobid.de] INFO: Scrape type for 3364735: 1
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365539%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [autobid.de] INFO: Scrape type for 3365539: 1
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364689%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [autobid.de] INFO: Scrape type for 3364689: 1
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364751%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [autobid.de] INFO: Scrape type for 3364751: 1
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365245%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [autobid.de] INFO: Scrape type for 3365245: 1
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365086%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [autobid.de] INFO: Scrape type for 3365086: 1
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364752%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [autobid.de] INFO: Scrape type for 3364752: 1
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364636%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [autobid.de] INFO: Scrape type for 3364636: 1
2026-04-01 06:29:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-a5-cabriolet-2-0-tdi-3364644/details> (referer: None)
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364635%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [autobid.de] INFO: Scrape type for 3364635: 1
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366396%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [autobid.de] INFO: Scrape type for 3366396: 1
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366506%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [autobid.de] INFO: Scrape type for 3366506: 1
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366507%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [autobid.de] INFO: Scrape type for 3366507: 1
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366508%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [autobid.de] INFO: Scrape type for 3366508: 1
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366505%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [autobid.de] INFO: Scrape type for 3366505: 1
2026-04-01 06:29:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 1356a3fc-2f21-5566-87ca-89d772f989e1 with identifier: 3364644
2026-04-01 06:29:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 1356a3fc-2f21-5566-87ca-89d772f989e1 with scrape type 1
2026-04-01 06:29:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 1356a3fc-2f21-5566-87ca-89d772f989e1
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364644%252F93541983_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364644/93541983_hd.jpg not downloaded yet
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366509%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [autobid.de] INFO: Scrape type for 3366509: 1
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364644%252F93541970_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364644/93541970_hd.jpg not downloaded yet
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366504%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [autobid.de] INFO: Scrape type for 3366504: 1
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364644%252F93541986_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364644/93541986_hd.jpg not downloaded yet
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366510%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [autobid.de] INFO: Scrape type for 3366510: 1
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364644%252F93541985_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364644/93541985_hd.jpg not downloaded yet
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364644%252F93541973_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364644/93541973_hd.jpg not downloaded yet
2026-04-01 06:29:35 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/seat-ibiza-1-4-sport-3364636/details> (referer: None)
2026-04-01 06:29:35 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 12272154-4234-5107-8c0b-c0047cc558b1 with identifier: 3364636
2026-04-01 06:29:35 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 12272154-4234-5107-8c0b-c0047cc558b1 with scrape type 1
2026-04-01 06:29:35 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 12272154-4234-5107-8c0b-c0047cc558b1
2026-04-01 06:29:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364636%252F93541634_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364636/93541634_hd.jpg not downloaded yet
2026-04-01 06:29:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364636%252F93541652_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364636/93541652_hd.jpg not downloaded yet
2026-04-01 06:29:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364636%252F93541651_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364636/93541651_hd.jpg not downloaded yet
2026-04-01 06:29:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364636%252F93541653_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364636/93541653_hd.jpg not downloaded yet
2026-04-01 06:29:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:29:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33647%252F3364636%252F93541642_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:29:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33647/3364636/93541642_hd.jpg not downloaded yet
2026-04-01 06:29:42 [scrapy.extensions.logstats] INFO: Crawled 135 pages (at 10 pages/min), scraped 6 items (at 0 items/min)
2026-04-01 06:29:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 234MiB
2026-04-01 06:30:42 [scrapy.extensions.logstats] INFO: Crawled 135 pages (at 0 pages/min), scraped 6 items (at 0 items/min)
2026-04-01 06:30:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 234MiB
2026-04-01 06:31:42 [scrapy.extensions.logstats] INFO: Crawled 135 pages (at 0 pages/min), scraped 6 items (at 0 items/min)
2026-04-01 06:31:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 234MiB
2026-04-01 06:32:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 1caede32-b60b-5a17-957d-7c034057e8f5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33666/3366554/93640931_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:32:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 1caede32-b60b-5a17-957d-7c034057e8f5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33666/3366554/93640920_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:32:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item c61183a1-512a-5144-a0dd-e3f29a627b7e due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33551/3355045/93459117_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:32:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item c61183a1-512a-5144-a0dd-e3f29a627b7e due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33551/3355045/93459116_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:32:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item c61183a1-512a-5144-a0dd-e3f29a627b7e due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33551/3355045/93459115_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:32:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 20144b9b-6632-5479-aca8-14b2667aa571 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33591/3359072/93528180_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:32:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 50fc0989-fe2d-5586-80c6-7e2254be6419 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33648/3364744/93546605_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:32:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=26> (referer: None)
2026-04-01 06:32:30 [autobid.de] INFO: Page 26/146: found 20 items
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360144%27 HTTP/1.1" 200 None
2026-04-01 06:32:30 [autobid.de] INFO: Scrape type for 3360144: 1
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359693%27 HTTP/1.1" 200 None
2026-04-01 06:32:30 [autobid.de] INFO: Scrape type for 3359693: 1
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360145%27 HTTP/1.1" 200 None
2026-04-01 06:32:30 [autobid.de] INFO: Scrape type for 3360145: 1
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359809%27 HTTP/1.1" 200 None
2026-04-01 06:32:30 [autobid.de] INFO: Scrape type for 3359809: 1
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360323%27 HTTP/1.1" 200 None
2026-04-01 06:32:30 [autobid.de] INFO: Scrape type for 3360323: 1
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363377%27 HTTP/1.1" 200 None
2026-04-01 06:32:30 [autobid.de] INFO: Scrape type for 3363377: 1
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360146%27 HTTP/1.1" 200 None
2026-04-01 06:32:30 [autobid.de] INFO: Scrape type for 3360146: 1
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273350585%27 HTTP/1.1" 200 None
2026-04-01 06:32:30 [autobid.de] INFO: Scrape type for 3350585: 1
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354911%27 HTTP/1.1" 200 None
2026-04-01 06:32:30 [autobid.de] INFO: Scrape type for 3354911: 1
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273344339%27 HTTP/1.1" 200 None
2026-04-01 06:32:30 [autobid.de] INFO: Scrape type for 3344339: 1
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345102%27 HTTP/1.1" 200 None
2026-04-01 06:32:30 [autobid.de] INFO: Scrape type for 3345102: 1
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345103%27 HTTP/1.1" 200 None
2026-04-01 06:32:30 [autobid.de] INFO: Scrape type for 3345103: 1
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359803%27 HTTP/1.1" 200 None
2026-04-01 06:32:30 [autobid.de] INFO: Scrape type for 3359803: 1
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273336652%27 HTTP/1.1" 200 None
2026-04-01 06:32:30 [autobid.de] INFO: Scrape type for 3336652: 1
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354139%27 HTTP/1.1" 200 None
2026-04-01 06:32:30 [autobid.de] INFO: Scrape type for 3354139: 1
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359793%27 HTTP/1.1" 200 None
2026-04-01 06:32:30 [autobid.de] INFO: Scrape type for 3359793: 1
2026-04-01 06:32:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359979%27 HTTP/1.1" 200 None
2026-04-01 06:32:31 [autobid.de] INFO: Scrape type for 3359979: 1
2026-04-01 06:32:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359801%27 HTTP/1.1" 200 None
2026-04-01 06:32:31 [autobid.de] INFO: Scrape type for 3359801: 1
2026-04-01 06:32:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359805%27 HTTP/1.1" 200 None
2026-04-01 06:32:31 [autobid.de] INFO: Scrape type for 3359805: 1
2026-04-01 06:32:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359797%27 HTTP/1.1" 200 None
2026-04-01 06:32:31 [autobid.de] INFO: Scrape type for 3359797: 1
2026-04-01 06:32:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mercedes-benz-sprinter-413-cdi-automatik-krankenwagen-3366510/details> (referer: None)
2026-04-01 06:32:31 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: dd2aa1ec-b4a7-50c1-8546-094ae99de019 with identifier: 3366510
2026-04-01 06:32:31 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item dd2aa1ec-b4a7-50c1-8546-094ae99de019 with scrape type 1
2026-04-01 06:32:31 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item dd2aa1ec-b4a7-50c1-8546-094ae99de019
2026-04-01 06:32:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33666%252F3366510%252F93634597_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:32:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33666/3366510/93634597_hd.jpg not downloaded yet
2026-04-01 06:32:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33666%252F3366510%252F93634589_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:32:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33666/3366510/93634589_hd.jpg not downloaded yet
2026-04-01 06:32:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33666%252F3366510%252F93634594_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:32:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33666/3366510/93634594_hd.jpg not downloaded yet
2026-04-01 06:32:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33666%252F3366510%252F93634612_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:32:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33666/3366510/93634612_hd.jpg not downloaded yet
2026-04-01 06:32:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33666%252F3366510%252F93634623_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:32:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33666/3366510/93634623_hd.jpg not downloaded yet
2026-04-01 06:32:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item b546be6e-ebde-53c8-93e6-bd5809aeb371 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33549/3354808/93545917_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:32:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=27> (referer: None)
2026-04-01 06:32:34 [autobid.de] INFO: Page 27/146: found 20 items
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359813%27 HTTP/1.1" 200 None
2026-04-01 06:32:34 [autobid.de] INFO: Scrape type for 3359813: 1
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359795%27 HTTP/1.1" 200 None
2026-04-01 06:32:34 [autobid.de] INFO: Scrape type for 3359795: 1
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359978%27 HTTP/1.1" 200 None
2026-04-01 06:32:34 [autobid.de] INFO: Scrape type for 3359978: 1
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359811%27 HTTP/1.1" 200 None
2026-04-01 06:32:34 [autobid.de] INFO: Scrape type for 3359811: 1
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345105%27 HTTP/1.1" 200 None
2026-04-01 06:32:34 [autobid.de] INFO: Scrape type for 3345105: 1
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345106%27 HTTP/1.1" 200 None
2026-04-01 06:32:34 [autobid.de] INFO: Scrape type for 3345106: 1
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345107%27 HTTP/1.1" 200 None
2026-04-01 06:32:34 [autobid.de] INFO: Scrape type for 3345107: 1
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359799%27 HTTP/1.1" 200 None
2026-04-01 06:32:34 [autobid.de] INFO: Scrape type for 3359799: 1
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359807%27 HTTP/1.1" 200 None
2026-04-01 06:32:34 [autobid.de] INFO: Scrape type for 3359807: 1
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359815%27 HTTP/1.1" 200 None
2026-04-01 06:32:34 [autobid.de] INFO: Scrape type for 3359815: 1
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345113%27 HTTP/1.1" 200 None
2026-04-01 06:32:34 [autobid.de] INFO: Scrape type for 3345113: 1
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345114%27 HTTP/1.1" 200 None
2026-04-01 06:32:34 [autobid.de] INFO: Scrape type for 3345114: 1
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345120%27 HTTP/1.1" 200 None
2026-04-01 06:32:34 [autobid.de] INFO: Scrape type for 3345120: 1
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345122%27 HTTP/1.1" 200 None
2026-04-01 06:32:34 [autobid.de] INFO: Scrape type for 3345122: 1
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345124%27 HTTP/1.1" 200 None
2026-04-01 06:32:34 [autobid.de] INFO: Scrape type for 3345124: 1
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345127%27 HTTP/1.1" 200 None
2026-04-01 06:32:34 [autobid.de] INFO: Scrape type for 3345127: 1
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361400%27 HTTP/1.1" 200 None
2026-04-01 06:32:34 [autobid.de] INFO: Scrape type for 3361400: 1
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363191%27 HTTP/1.1" 200 None
2026-04-01 06:32:34 [autobid.de] INFO: Scrape type for 3363191: 1
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357502%27 HTTP/1.1" 200 None
2026-04-01 06:32:34 [autobid.de] INFO: Scrape type for 3357502: 1
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:32:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360131%27 HTTP/1.1" 200 None
2026-04-01 06:32:34 [autobid.de] INFO: Scrape type for 3360131: 1
2026-04-01 06:32:42 [scrapy.extensions.logstats] INFO: Crawled 138 pages (at 3 pages/min), scraped 6 items (at 0 items/min)
2026-04-01 06:32:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 234MiB
2026-04-01 06:33:42 [scrapy.extensions.logstats] INFO: Crawled 138 pages (at 0 pages/min), scraped 6 items (at 0 items/min)
2026-04-01 06:33:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 234MiB
2026-04-01 06:34:42 [scrapy.extensions.logstats] INFO: Crawled 138 pages (at 0 pages/min), scraped 6 items (at 0 items/min)
2026-04-01 06:34:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 234MiB
2026-04-01 06:35:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 4770e7c0-e5ef-5288-95bc-08bcd0b0bbc6 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33670/3366959/93652354_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:35:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 976d2125-f0b4-5b61-a20f-a395fafc9110 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33652/3365150/93565088_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:35:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/citroen-c3-1-2-puretech-feel-3360131/details> (referer: None)
2026-04-01 06:35:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: fea49495-c294-56d3-ada3-d6401457f81e with identifier: 3360131
2026-04-01 06:35:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item fea49495-c294-56d3-ada3-d6401457f81e with scrape type 1
2026-04-01 06:35:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item fea49495-c294-56d3-ada3-d6401457f81e
2026-04-01 06:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33602%252F3360131%252F93491044_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33602/3360131/93491044_hd.jpg not downloaded yet
2026-04-01 06:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33602%252F3360131%252F93491045_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33602/3360131/93491045_hd.jpg not downloaded yet
2026-04-01 06:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33602%252F3360131%252F93491046_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33602/3360131/93491046_hd.jpg not downloaded yet
2026-04-01 06:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33602%252F3360131%252F93491047_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33602/3360131/93491047_hd.jpg not downloaded yet
2026-04-01 06:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33602%252F3360131%252F93491048_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33602/3360131/93491048_hd.jpg not downloaded yet
2026-04-01 06:35:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=28> (referer: None)
2026-04-01 06:35:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/renault-zoe-intens-electric-3357502/details> (referer: None)
2026-04-01 06:35:22 [autobid.de] INFO: Page 28/146: found 20 items
2026-04-01 06:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355472%27 HTTP/1.1" 200 None
2026-04-01 06:35:22 [autobid.de] INFO: Scrape type for 3355472: 1
2026-04-01 06:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360485%27 HTTP/1.1" 200 None
2026-04-01 06:35:22 [autobid.de] INFO: Scrape type for 3360485: 1
2026-04-01 06:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355189%27 HTTP/1.1" 200 None
2026-04-01 06:35:22 [autobid.de] INFO: Scrape type for 3355189: 1
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360357%27 HTTP/1.1" 200 None
2026-04-01 06:35:23 [autobid.de] INFO: Scrape type for 3360357: 1
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355207%27 HTTP/1.1" 200 None
2026-04-01 06:35:23 [autobid.de] INFO: Scrape type for 3355207: 1
2026-04-01 06:35:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: dbd74cad-a97a-5f61-9478-5e10d54d8f0a with identifier: 3357502
2026-04-01 06:35:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item dbd74cad-a97a-5f61-9478-5e10d54d8f0a with scrape type 1
2026-04-01 06:35:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item dbd74cad-a97a-5f61-9478-5e10d54d8f0a
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357502%252F93536869_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357502/93536869_hd.jpg not downloaded yet
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358328%27 HTTP/1.1" 200 None
2026-04-01 06:35:23 [autobid.de] INFO: Scrape type for 3358328: 1
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357502%252F93536870_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357502/93536870_hd.jpg not downloaded yet
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360356%27 HTTP/1.1" 200 None
2026-04-01 06:35:23 [autobid.de] INFO: Scrape type for 3360356: 1
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357502%252F93536883_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357502/93536883_hd.jpg not downloaded yet
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355213%27 HTTP/1.1" 200 None
2026-04-01 06:35:23 [autobid.de] INFO: Scrape type for 3355213: 1
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357502%252F93536884_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357502/93536884_hd.jpg not downloaded yet
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360486%27 HTTP/1.1" 200 None
2026-04-01 06:35:23 [autobid.de] INFO: Scrape type for 3360486: 1
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33576%252F3357502%252F93536877_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33576/3357502/93536877_hd.jpg not downloaded yet
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355210%27 HTTP/1.1" 200 None
2026-04-01 06:35:23 [autobid.de] INFO: Scrape type for 3355210: 1
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360361%27 HTTP/1.1" 200 None
2026-04-01 06:35:23 [autobid.de] INFO: Scrape type for 3360361: 1
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355214%27 HTTP/1.1" 200 None
2026-04-01 06:35:23 [autobid.de] INFO: Scrape type for 3355214: 1
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360113%27 HTTP/1.1" 200 None
2026-04-01 06:35:23 [autobid.de] INFO: Scrape type for 3360113: 1
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355203%27 HTTP/1.1" 200 None
2026-04-01 06:35:23 [autobid.de] INFO: Scrape type for 3355203: 1
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357163%27 HTTP/1.1" 200 None
2026-04-01 06:35:23 [autobid.de] INFO: Scrape type for 3357163: 1
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355208%27 HTTP/1.1" 200 None
2026-04-01 06:35:23 [autobid.de] INFO: Scrape type for 3355208: 1
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358329%27 HTTP/1.1" 200 None
2026-04-01 06:35:23 [autobid.de] INFO: Scrape type for 3358329: 1
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355192%27 HTTP/1.1" 200 None
2026-04-01 06:35:23 [autobid.de] INFO: Scrape type for 3355192: 1
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358326%27 HTTP/1.1" 200 None
2026-04-01 06:35:23 [autobid.de] INFO: Scrape type for 3358326: 1
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355209%27 HTTP/1.1" 200 None
2026-04-01 06:35:23 [autobid.de] INFO: Scrape type for 3355209: 1
2026-04-01 06:35:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 976d2125-f0b4-5b61-a20f-a395fafc9110 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33652/3365150/93565104_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:35:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 976d2125-f0b4-5b61-a20f-a395fafc9110 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33652/3365150/93565105_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:35:25 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mercedes-benz-c-220-d-t-automatik-3355209/details> (referer: None)
2026-04-01 06:35:26 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 545a857b-5af4-579d-a0dc-c89c834b572c with identifier: 3355209
2026-04-01 06:35:26 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 545a857b-5af4-579d-a0dc-c89c834b572c with scrape type 1
2026-04-01 06:35:26 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 545a857b-5af4-579d-a0dc-c89c834b572c
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33553%252F3355209%252F93282264_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33553/3355209/93282264_hd.jpg not downloaded yet
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33553%252F3355209%252F93282265_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33553/3355209/93282265_hd.jpg not downloaded yet
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33553%252F3355209%252F93282266_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33553/3355209/93282266_hd.jpg not downloaded yet
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33553%252F3355209%252F93282267_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33553/3355209/93282267_hd.jpg not downloaded yet
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33553%252F3355209%252F93282268_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33553/3355209/93282268_hd.jpg not downloaded yet
2026-04-01 06:35:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=29> (referer: None)
2026-04-01 06:35:26 [autobid.de] INFO: Page 29/146: found 20 items
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360114%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [autobid.de] INFO: Scrape type for 3360114: 1
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355198%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [autobid.de] INFO: Scrape type for 3355198: 1
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360132%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [autobid.de] INFO: Scrape type for 3360132: 1
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355206%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [autobid.de] INFO: Scrape type for 3355206: 1
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360355%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [autobid.de] INFO: Scrape type for 3360355: 1
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355204%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [autobid.de] INFO: Scrape type for 3355204: 1
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360362%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [autobid.de] INFO: Scrape type for 3360362: 1
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355199%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [autobid.de] INFO: Scrape type for 3355199: 1
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358667%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [autobid.de] INFO: Scrape type for 3358667: 1
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355211%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [autobid.de] INFO: Scrape type for 3355211: 1
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360358%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [autobid.de] INFO: Scrape type for 3360358: 1
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355191%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [autobid.de] INFO: Scrape type for 3355191: 1
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360359%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [autobid.de] INFO: Scrape type for 3360359: 1
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355194%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [autobid.de] INFO: Scrape type for 3355194: 1
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358330%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [autobid.de] INFO: Scrape type for 3358330: 1
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355197%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [autobid.de] INFO: Scrape type for 3355197: 1
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360133%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [autobid.de] INFO: Scrape type for 3360133: 1
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355195%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [autobid.de] INFO: Scrape type for 3355195: 1
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358668%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [autobid.de] INFO: Scrape type for 3358668: 1
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355200%27 HTTP/1.1" 200 None
2026-04-01 06:35:26 [autobid.de] INFO: Scrape type for 3355200: 1
2026-04-01 06:35:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 70ce78c2-b4ae-5b12-8981-d91ab9a9e6c8 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33653/3365263/93580303_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:35:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-polo-1-6i-automatik-united-3358326/details> (referer: None)
2026-04-01 06:35:27 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: caf15a99-6498-5e18-9d4b-9aeb6b691203 with identifier: 3358326
2026-04-01 06:35:27 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item caf15a99-6498-5e18-9d4b-9aeb6b691203 with scrape type 1
2026-04-01 06:35:27 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item caf15a99-6498-5e18-9d4b-9aeb6b691203
2026-04-01 06:35:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33584%252F3358326%252F93371427_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33584/3358326/93371427_hd.jpg not downloaded yet
2026-04-01 06:35:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33584%252F3358326%252F93371430_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33584/3358326/93371430_hd.jpg not downloaded yet
2026-04-01 06:35:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33584%252F3358326%252F93371436_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33584/3358326/93371436_hd.jpg not downloaded yet
2026-04-01 06:35:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33584%252F3358326%252F93371434_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33584/3358326/93371434_hd.jpg not downloaded yet
2026-04-01 06:35:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33584%252F3358326%252F93371432_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33584/3358326/93371432_hd.jpg not downloaded yet
2026-04-01 06:35:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 6b57c858-513a-5f76-a56e-7b962433d48c due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33653/3365262/93580260_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:35:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-q3-35-tfsi-s-line-3355200/details> (referer: None)
2026-04-01 06:35:29 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 0e89bb76-52b2-5521-baee-cd1cf7345e3d with identifier: 3355200
2026-04-01 06:35:29 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 0e89bb76-52b2-5521-baee-cd1cf7345e3d with scrape type 1
2026-04-01 06:35:29 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 0e89bb76-52b2-5521-baee-cd1cf7345e3d
2026-04-01 06:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33553%252F3355200%252F93334050_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33553/3355200/93334050_hd.jpg not downloaded yet
2026-04-01 06:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33553%252F3355200%252F93334048_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33553/3355200/93334048_hd.jpg not downloaded yet
2026-04-01 06:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33553%252F3355200%252F93334055_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33553/3355200/93334055_hd.jpg not downloaded yet
2026-04-01 06:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33553%252F3355200%252F93334052_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33553/3355200/93334052_hd.jpg not downloaded yet
2026-04-01 06:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33553%252F3355200%252F93334053_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33553/3355200/93334053_hd.jpg not downloaded yet
2026-04-01 06:35:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 6b57c858-513a-5f76-a56e-7b962433d48c due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33653/3365262/93580255_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:35:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=30> (referer: None)
2026-04-01 06:35:30 [autobid.de] INFO: Page 30/146: found 20 items
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353949%27 HTTP/1.1" 200 None
2026-04-01 06:35:30 [autobid.de] INFO: Scrape type for 3353949: 1
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362797%27 HTTP/1.1" 200 None
2026-04-01 06:35:30 [autobid.de] INFO: Scrape type for 3362797: 1
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358970%27 HTTP/1.1" 200 None
2026-04-01 06:35:30 [autobid.de] INFO: Scrape type for 3358970: 1
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362625%27 HTTP/1.1" 200 None
2026-04-01 06:35:30 [autobid.de] INFO: Scrape type for 3362625: 1
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358971%27 HTTP/1.1" 200 None
2026-04-01 06:35:30 [autobid.de] INFO: Scrape type for 3358971: 1
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362786%27 HTTP/1.1" 200 None
2026-04-01 06:35:30 [autobid.de] INFO: Scrape type for 3362786: 1
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362626%27 HTTP/1.1" 200 None
2026-04-01 06:35:30 [autobid.de] INFO: Scrape type for 3362626: 1
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358973%27 HTTP/1.1" 200 None
2026-04-01 06:35:30 [autobid.de] INFO: Scrape type for 3358973: 1
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362792%27 HTTP/1.1" 200 None
2026-04-01 06:35:30 [autobid.de] INFO: Scrape type for 3362792: 1
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358974%27 HTTP/1.1" 200 None
2026-04-01 06:35:30 [autobid.de] INFO: Scrape type for 3358974: 1
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362624%27 HTTP/1.1" 200 None
2026-04-01 06:35:30 [autobid.de] INFO: Scrape type for 3362624: 1
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358975%27 HTTP/1.1" 200 None
2026-04-01 06:35:30 [autobid.de] INFO: Scrape type for 3358975: 1
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362783%27 HTTP/1.1" 200 None
2026-04-01 06:35:30 [autobid.de] INFO: Scrape type for 3362783: 1
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358976%27 HTTP/1.1" 200 None
2026-04-01 06:35:30 [autobid.de] INFO: Scrape type for 3358976: 1
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362622%27 HTTP/1.1" 200 None
2026-04-01 06:35:30 [autobid.de] INFO: Scrape type for 3362622: 1
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358977%27 HTTP/1.1" 200 None
2026-04-01 06:35:30 [autobid.de] INFO: Scrape type for 3358977: 1
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362798%27 HTTP/1.1" 200 None
2026-04-01 06:35:30 [autobid.de] INFO: Scrape type for 3362798: 1
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358978%27 HTTP/1.1" 200 None
2026-04-01 06:35:30 [autobid.de] INFO: Scrape type for 3358978: 1
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362790%27 HTTP/1.1" 200 None
2026-04-01 06:35:30 [autobid.de] INFO: Scrape type for 3362790: 1
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358979%27 HTTP/1.1" 200 None
2026-04-01 06:35:30 [autobid.de] INFO: Scrape type for 3358979: 1
2026-04-01 06:35:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 1356a3fc-2f21-5566-87ca-89d772f989e1 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33647/3364644/93541973_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:35:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=31> (referer: None)
2026-04-01 06:35:35 [autobid.de] INFO: Page 31/146: found 20 items
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362789%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [autobid.de] INFO: Scrape type for 3362789: 1
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358980%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [autobid.de] INFO: Scrape type for 3358980: 1
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362800%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [autobid.de] INFO: Scrape type for 3362800: 1
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358981%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [autobid.de] INFO: Scrape type for 3358981: 1
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362795%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [autobid.de] INFO: Scrape type for 3362795: 1
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358982%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [autobid.de] INFO: Scrape type for 3358982: 1
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362793%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [autobid.de] INFO: Scrape type for 3362793: 1
2026-04-01 06:35:35 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/suzuki-grand-vitara-1-9-ddis-comfort-3358979/details> (referer: None)
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358983%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [autobid.de] INFO: Scrape type for 3358983: 1
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362796%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [autobid.de] INFO: Scrape type for 3362796: 1
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358984%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [autobid.de] INFO: Scrape type for 3358984: 1
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362623%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [autobid.de] INFO: Scrape type for 3362623: 1
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358985%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [autobid.de] INFO: Scrape type for 3358985: 1
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362794%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [autobid.de] INFO: Scrape type for 3362794: 1
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358986%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [autobid.de] INFO: Scrape type for 3358986: 1
2026-04-01 06:35:35 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: fef9fca6-601b-59c3-9f8f-fc898f07fa63 with identifier: 3358979
2026-04-01 06:35:35 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item fef9fca6-601b-59c3-9f8f-fc898f07fa63 with scrape type 1
2026-04-01 06:35:35 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item fef9fca6-601b-59c3-9f8f-fc898f07fa63
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33590%252F3358979%252F93519479_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33590/3358979/93519479_hd.jpg not downloaded yet
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362799%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [autobid.de] INFO: Scrape type for 3362799: 1
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33590%252F3358979%252F93519746_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33590/3358979/93519746_hd.jpg not downloaded yet
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358987%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [autobid.de] INFO: Scrape type for 3358987: 1
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33590%252F3358979%252F93519780_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33590/3358979/93519780_hd.jpg not downloaded yet
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362791%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [autobid.de] INFO: Scrape type for 3362791: 1
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33590%252F3358979%252F93519765_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33590/3358979/93519765_hd.jpg not downloaded yet
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358988%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [autobid.de] INFO: Scrape type for 3358988: 1
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33590%252F3358979%252F93519415_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33590/3358979/93519415_hd.jpg not downloaded yet
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362788%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [autobid.de] INFO: Scrape type for 3362788: 1
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358989%27 HTTP/1.1" 200 None
2026-04-01 06:35:35 [autobid.de] INFO: Scrape type for 3358989: 1
2026-04-01 06:35:36 [scrapy.core.engine] DEBUG: Crawled (403) <GET https://autobid.de/en/item/vw-t7-california-lu-2-0-tdi-dsg-ocean-3362793/details> (referer: None)
2026-04-01 06:35:36 [crawlers.middlewares.monitoring_spider_middleware] ERROR: Ignoring non-200 response
NoneType: None
2026-04-01 06:35:37 [scrapy.core.scraper] ERROR: Spider error processing <GET https://autobid.de/en/item/vw-t7-california-lu-2-0-tdi-dsg-ocean-3362793/details> (referer: None)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/twisted/internet/defer.py", line 1078, in _runCallbacks
current.result = callback( # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 301, in process_spider_exception
return self._process_spider_exception(response, spider, _failure)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 150, in _process_spider_exception
result = method(response=response, exception=exception, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/crawlers/middlewares/monitoring_spider_middleware.py", line 73, in process_spider_exception
raise exception
File "/usr/local/lib/python3.11/dist-packages/scrapy/core/spidermw.py", line 83, in _process_spider_input
result = method(response=response, spider=spider)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/scrapy/spidermiddlewares/httperror.py", line 46, in process_spider_input
raise HttpError(response, "Ignoring non-200 response")
scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response
2026-04-01 06:35:39 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=32> (referer: None)
2026-04-01 06:35:39 [autobid.de] INFO: Page 32/146: found 20 items
2026-04-01 06:35:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358990%27 HTTP/1.1" 200 None
2026-04-01 06:35:39 [autobid.de] INFO: Scrape type for 3358990: 1
2026-04-01 06:35:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358991%27 HTTP/1.1" 200 None
2026-04-01 06:35:39 [autobid.de] INFO: Scrape type for 3358991: 1
2026-04-01 06:35:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358992%27 HTTP/1.1" 200 None
2026-04-01 06:35:39 [autobid.de] INFO: Scrape type for 3358992: 1
2026-04-01 06:35:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358993%27 HTTP/1.1" 200 None
2026-04-01 06:35:39 [autobid.de] INFO: Scrape type for 3358993: 1
2026-04-01 06:35:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358994%27 HTTP/1.1" 200 None
2026-04-01 06:35:40 [autobid.de] INFO: Scrape type for 3358994: 1
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358995%27 HTTP/1.1" 200 None
2026-04-01 06:35:40 [autobid.de] INFO: Scrape type for 3358995: 1
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358997%27 HTTP/1.1" 200 None
2026-04-01 06:35:40 [autobid.de] INFO: Scrape type for 3358997: 1
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358998%27 HTTP/1.1" 200 None
2026-04-01 06:35:40 [autobid.de] INFO: Scrape type for 3358998: 1
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358999%27 HTTP/1.1" 200 None
2026-04-01 06:35:40 [autobid.de] INFO: Scrape type for 3358999: 1
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359000%27 HTTP/1.1" 200 None
2026-04-01 06:35:40 [autobid.de] INFO: Scrape type for 3359000: 1
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359001%27 HTTP/1.1" 200 None
2026-04-01 06:35:40 [autobid.de] INFO: Scrape type for 3359001: 1
2026-04-01 06:35:40 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/renault-twingo-0-9-tce-limeted-3358989/details> (referer: None)
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359002%27 HTTP/1.1" 200 None
2026-04-01 06:35:40 [autobid.de] INFO: Scrape type for 3359002: 1
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359003%27 HTTP/1.1" 200 None
2026-04-01 06:35:40 [autobid.de] INFO: Scrape type for 3359003: 1
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359004%27 HTTP/1.1" 200 None
2026-04-01 06:35:40 [autobid.de] INFO: Scrape type for 3359004: 1
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359005%27 HTTP/1.1" 200 None
2026-04-01 06:35:40 [autobid.de] INFO: Scrape type for 3359005: 1
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359006%27 HTTP/1.1" 200 None
2026-04-01 06:35:40 [autobid.de] INFO: Scrape type for 3359006: 1
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359007%27 HTTP/1.1" 200 None
2026-04-01 06:35:40 [autobid.de] INFO: Scrape type for 3359007: 1
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359008%27 HTTP/1.1" 200 None
2026-04-01 06:35:40 [autobid.de] INFO: Scrape type for 3359008: 1
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359009%27 HTTP/1.1" 200 None
2026-04-01 06:35:40 [autobid.de] INFO: Scrape type for 3359009: 1
2026-04-01 06:35:40 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: c09fb3ae-9417-560b-8024-44d3255011e5 with identifier: 3358989
2026-04-01 06:35:40 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item c09fb3ae-9417-560b-8024-44d3255011e5 with scrape type 1
2026-04-01 06:35:40 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item c09fb3ae-9417-560b-8024-44d3255011e5
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33590%252F3358989%252F93485607_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:40 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33590/3358989/93485607_hd.jpg not downloaded yet
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359010%27 HTTP/1.1" 200 None
2026-04-01 06:35:40 [autobid.de] INFO: Scrape type for 3359010: 1
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33590%252F3358989%252F93485611_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:40 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33590/3358989/93485611_hd.jpg not downloaded yet
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33590%252F3358989%252F93485619_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:40 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33590/3358989/93485619_hd.jpg not downloaded yet
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33590%252F3358989%252F93485615_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:40 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33590/3358989/93485615_hd.jpg not downloaded yet
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33590%252F3358989%252F93485613_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:40 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33590/3358989/93485613_hd.jpg not downloaded yet
2026-04-01 06:35:41 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ford-fiesta-1-3-ambiente-3359001/details> (referer: None)
2026-04-01 06:35:41 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 2a0cd082-6c1b-5552-a1b0-7790e999ea8e with identifier: 3359001
2026-04-01 06:35:41 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 2a0cd082-6c1b-5552-a1b0-7790e999ea8e with scrape type 1
2026-04-01 06:35:41 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 2a0cd082-6c1b-5552-a1b0-7790e999ea8e
2026-04-01 06:35:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359001%252F93487134_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:41 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359001/93487134_hd.jpg not downloaded yet
2026-04-01 06:35:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359001%252F93487139_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:41 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359001/93487139_hd.jpg not downloaded yet
2026-04-01 06:35:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359001%252F93487147_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:41 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359001/93487147_hd.jpg not downloaded yet
2026-04-01 06:35:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359001%252F93487143_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:41 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359001/93487143_hd.jpg not downloaded yet
2026-04-01 06:35:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:35:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359001%252F93487131_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:35:41 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359001/93487131_hd.jpg not downloaded yet
2026-04-01 06:35:42 [scrapy.extensions.logstats] INFO: Crawled 152 pages (at 14 pages/min), scraped 6 items (at 0 items/min)
2026-04-01 06:35:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 237MiB
2026-04-01 06:36:42 [scrapy.extensions.logstats] INFO: Crawled 152 pages (at 0 pages/min), scraped 6 items (at 0 items/min)
2026-04-01 06:36:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 237MiB
2026-04-01 06:37:42 [scrapy.extensions.logstats] INFO: Crawled 152 pages (at 0 pages/min), scraped 6 items (at 0 items/min)
2026-04-01 06:37:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 237MiB
2026-04-01 06:38:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 12272154-4234-5107-8c0b-c0047cc558b1 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33647/3364636/93541642_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:38:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 12272154-4234-5107-8c0b-c0047cc558b1 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33647/3364636/93541653_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:38:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 12272154-4234-5107-8c0b-c0047cc558b1 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33647/3364636/93541651_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:38:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 12272154-4234-5107-8c0b-c0047cc558b1 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33647/3364636/93541652_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:38:27 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 12272154-4234-5107-8c0b-c0047cc558b1 with identifier: 3364636
2026-04-01 06:38:27 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 12272154-4234-5107-8c0b-c0047cc558b1 with identifier: 3364636
2026-04-01 06:38:27 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: 12272154-4234-5107-8c0b-c0047cc558b1 with identifier: 3364636 to the API
2026-04-01 06:38:27 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 06:38:28 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 06:38:28 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33647/3364636/93541634_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 81314',
'body_type': 'Saloon',
'category': 'Saloon',
'color': 'black Pearl effect',
'currency': 'EUR',
'emission_standard': 'EURO 4',
'engine': '1.4L',
'engine_horse_power': 86,
'expiration_date': '2026-04-01T09:30:00Z',
'fuel_type': 'Petrol',
'id': '12272154-4234-5107-8c0b-c0047cc558b1',
'is_damaged': False,
'is_operable': True,
'is_vat_deductible': False,
'link_to_web_offer': 'https://autobid.de/en/item/seat-ibiza-1-4-sport-3364636',
'listing': 'auction',
'location': 'Landshut',
'make': 'Seat',
'model': 'Ibiza',
'odometer': 243100,
'odometer_reading_unit': 'km',
'origin_country_code': 'DE',
'original_photo_count': 5,
'photo_urls': ['https://cdn.autobid.de/data/cars/33647/3364636/93541634_hd.jpg',
'https://cdn.autobid.de/data/cars/33647/3364636/93541652_hd.jpg',
'https://cdn.autobid.de/data/cars/33647/3364636/93541651_hd.jpg',
'https://cdn.autobid.de/data/cars/33647/3364636/93541653_hd.jpg',
'https://cdn.autobid.de/data/cars/33647/3364636/93541642_hd.jpg'],
'photos': [],
'price': 500,
'price_includes_vat': False,
'registration_date': '2007-12-01',
'seller_name': 'Subsidiary/authorised dealer of the brand(s): VW',
'title': 'Seat Ibiza 1,4 "Sport"',
'transmission': 'manual',
'vin': None,
'year': 2007}
2026-04-01 06:38:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:38:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364636%27 HTTP/1.1" 200 None
2026-04-01 06:38:28 [autobid.de] INFO: Saving data for 3364636: {'created_time': 1775025508.139104, 'last_price_update_time': 1775025508.139119, 'auction_closing_time': 1775035800.0}
2026-04-01 06:38:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:38:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3364636') HTTP/1.1" 204 0
2026-04-01 06:38:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 12272154-4234-5107-8c0b-c0047cc558b1 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33647/3364636/93541634_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:38:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 1356a3fc-2f21-5566-87ca-89d772f989e1 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33647/3364644/93541985_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:38:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item dd2aa1ec-b4a7-50c1-8546-094ae99de019 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33666/3366510/93634623_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:38:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item dd2aa1ec-b4a7-50c1-8546-094ae99de019 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33666/3366510/93634612_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:38:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=33> (referer: None)
2026-04-01 06:38:34 [autobid.de] INFO: Page 33/146: found 20 items
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359014%27 HTTP/1.1" 200 None
2026-04-01 06:38:34 [autobid.de] INFO: Scrape type for 3359014: 1
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359015%27 HTTP/1.1" 200 None
2026-04-01 06:38:34 [autobid.de] INFO: Scrape type for 3359015: 1
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359016%27 HTTP/1.1" 200 None
2026-04-01 06:38:34 [autobid.de] INFO: Scrape type for 3359016: 1
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359018%27 HTTP/1.1" 200 None
2026-04-01 06:38:34 [autobid.de] INFO: Scrape type for 3359018: 1
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359019%27 HTTP/1.1" 200 None
2026-04-01 06:38:34 [autobid.de] INFO: Scrape type for 3359019: 1
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359020%27 HTTP/1.1" 200 None
2026-04-01 06:38:34 [autobid.de] INFO: Scrape type for 3359020: 1
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359021%27 HTTP/1.1" 200 None
2026-04-01 06:38:34 [autobid.de] INFO: Scrape type for 3359021: 1
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359022%27 HTTP/1.1" 200 None
2026-04-01 06:38:34 [autobid.de] INFO: Scrape type for 3359022: 1
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359023%27 HTTP/1.1" 200 None
2026-04-01 06:38:34 [autobid.de] INFO: Scrape type for 3359023: 1
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359024%27 HTTP/1.1" 200 None
2026-04-01 06:38:34 [autobid.de] INFO: Scrape type for 3359024: 1
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359025%27 HTTP/1.1" 200 None
2026-04-01 06:38:34 [autobid.de] INFO: Scrape type for 3359025: 1
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359026%27 HTTP/1.1" 200 None
2026-04-01 06:38:34 [autobid.de] INFO: Scrape type for 3359026: 1
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359027%27 HTTP/1.1" 200 None
2026-04-01 06:38:34 [autobid.de] INFO: Scrape type for 3359027: 1
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359028%27 HTTP/1.1" 200 None
2026-04-01 06:38:34 [autobid.de] INFO: Scrape type for 3359028: 1
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359029%27 HTTP/1.1" 200 None
2026-04-01 06:38:34 [autobid.de] INFO: Scrape type for 3359029: 1
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359030%27 HTTP/1.1" 200 None
2026-04-01 06:38:34 [autobid.de] INFO: Scrape type for 3359030: 1
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359031%27 HTTP/1.1" 200 None
2026-04-01 06:38:34 [autobid.de] INFO: Scrape type for 3359031: 1
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359032%27 HTTP/1.1" 200 None
2026-04-01 06:38:34 [autobid.de] INFO: Scrape type for 3359032: 1
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359033%27 HTTP/1.1" 200 None
2026-04-01 06:38:34 [autobid.de] INFO: Scrape type for 3359033: 1
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:38:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359034%27 HTTP/1.1" 200 None
2026-04-01 06:38:34 [autobid.de] INFO: Scrape type for 3359034: 1
2026-04-01 06:38:42 [scrapy.extensions.logstats] INFO: Crawled 153 pages (at 1 pages/min), scraped 7 items (at 1 items/min)
2026-04-01 06:38:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 237MiB
2026-04-01 06:39:42 [scrapy.extensions.logstats] INFO: Crawled 153 pages (at 0 pages/min), scraped 7 items (at 0 items/min)
2026-04-01 06:39:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 237MiB
2026-04-01 06:40:42 [scrapy.extensions.logstats] INFO: Crawled 153 pages (at 0 pages/min), scraped 7 items (at 0 items/min)
2026-04-01 06:40:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 237MiB
2026-04-01 06:41:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item fea49495-c294-56d3-ada3-d6401457f81e due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33602/3360131/93491048_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:41:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item fea49495-c294-56d3-ada3-d6401457f81e due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33602/3360131/93491047_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:41:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-a4-avant-40-tdi-s-tronic-3359034/details> (referer: None)
2026-04-01 06:41:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: f92d4fd0-f2f4-5167-8590-8960c79d4dbb with identifier: 3359034
2026-04-01 06:41:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f92d4fd0-f2f4-5167-8590-8960c79d4dbb with scrape type 1
2026-04-01 06:41:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item f92d4fd0-f2f4-5167-8590-8960c79d4dbb
2026-04-01 06:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359034%252F93506300_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359034/93506300_hd.jpg not downloaded yet
2026-04-01 06:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359034%252F93506303_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359034/93506303_hd.jpg not downloaded yet
2026-04-01 06:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359034%252F93506323_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359034/93506323_hd.jpg not downloaded yet
2026-04-01 06:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359034%252F93506313_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359034/93506313_hd.jpg not downloaded yet
2026-04-01 06:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359034%252F93506308_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359034/93506308_hd.jpg not downloaded yet
2026-04-01 06:41:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=34> (referer: None)
2026-04-01 06:41:22 [autobid.de] INFO: Page 34/145: found 20 items
2026-04-01 06:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359038%27 HTTP/1.1" 200 None
2026-04-01 06:41:22 [autobid.de] INFO: Scrape type for 3359038: 1
2026-04-01 06:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359039%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [autobid.de] INFO: Scrape type for 3359039: 1
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359040%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [autobid.de] INFO: Scrape type for 3359040: 1
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359041%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [autobid.de] INFO: Scrape type for 3359041: 1
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359042%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [autobid.de] INFO: Scrape type for 3359042: 1
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359043%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [autobid.de] INFO: Scrape type for 3359043: 1
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359046%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [autobid.de] INFO: Scrape type for 3359046: 1
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359047%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [autobid.de] INFO: Scrape type for 3359047: 1
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360074%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [autobid.de] INFO: Scrape type for 3360074: 1
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362013%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [autobid.de] INFO: Scrape type for 3362013: 1
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362014%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [autobid.de] INFO: Scrape type for 3362014: 1
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362015%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [autobid.de] INFO: Scrape type for 3362015: 1
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362016%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [autobid.de] INFO: Scrape type for 3362016: 1
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362017%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [autobid.de] INFO: Scrape type for 3362017: 1
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362018%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [autobid.de] INFO: Scrape type for 3362018: 1
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362019%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [autobid.de] INFO: Scrape type for 3362019: 1
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362020%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [autobid.de] INFO: Scrape type for 3362020: 1
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362022%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [autobid.de] INFO: Scrape type for 3362022: 1
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362023%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [autobid.de] INFO: Scrape type for 3362023: 1
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362024%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [autobid.de] INFO: Scrape type for 3362024: 1
2026-04-01 06:41:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-q8-50-tdi-quattro-tiptronic-competition-plus-3359033/details> (referer: None)
2026-04-01 06:41:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 3b9e2237-3016-5051-948a-3584005cad82 with identifier: 3359033
2026-04-01 06:41:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 3b9e2237-3016-5051-948a-3584005cad82 with scrape type 1
2026-04-01 06:41:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 3b9e2237-3016-5051-948a-3584005cad82
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359033%252F93491011_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359033/93491011_hd.jpg not downloaded yet
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359033%252F93491015_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359033/93491015_hd.jpg not downloaded yet
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359033%252F93491022_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359033/93491022_hd.jpg not downloaded yet
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359033%252F93491019_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359033/93491019_hd.jpg not downloaded yet
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33591%252F3359033%252F93491017_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33591/3359033/93491017_hd.jpg not downloaded yet
2026-04-01 06:41:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 545a857b-5af4-579d-a0dc-c89c834b572c due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33553/3355209/93282268_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:41:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item caf15a99-6498-5e18-9d4b-9aeb6b691203 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33584/3358326/93371432_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:41:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=35> (referer: None)
2026-04-01 06:41:26 [autobid.de] INFO: Page 35/145: found 20 items
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356733%27 HTTP/1.1" 200 None
2026-04-01 06:41:26 [autobid.de] INFO: Scrape type for 3356733: 1
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273348104%27 HTTP/1.1" 200 None
2026-04-01 06:41:26 [autobid.de] INFO: Scrape type for 3348104: 1
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273351583%27 HTTP/1.1" 200 None
2026-04-01 06:41:26 [autobid.de] INFO: Scrape type for 3351583: 1
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357533%27 HTTP/1.1" 200 None
2026-04-01 06:41:26 [autobid.de] INFO: Scrape type for 3357533: 1
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359137%27 HTTP/1.1" 200 None
2026-04-01 06:41:26 [autobid.de] INFO: Scrape type for 3359137: 1
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359278%27 HTTP/1.1" 200 None
2026-04-01 06:41:26 [autobid.de] INFO: Scrape type for 3359278: 1
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273336712%27 HTTP/1.1" 200 None
2026-04-01 06:41:26 [autobid.de] INFO: Scrape type for 3336712: 1
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360360%27 HTTP/1.1" 200 None
2026-04-01 06:41:26 [autobid.de] INFO: Scrape type for 3360360: 1
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360687%27 HTTP/1.1" 200 None
2026-04-01 06:41:26 [autobid.de] INFO: Scrape type for 3360687: 1
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273335105%27 HTTP/1.1" 200 None
2026-04-01 06:41:26 [autobid.de] INFO: Scrape type for 3335105: 1
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360688%27 HTTP/1.1" 200 None
2026-04-01 06:41:26 [autobid.de] INFO: Scrape type for 3360688: 1
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360689%27 HTTP/1.1" 200 None
2026-04-01 06:41:26 [autobid.de] INFO: Scrape type for 3360689: 1
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360690%27 HTTP/1.1" 200 None
2026-04-01 06:41:26 [autobid.de] INFO: Scrape type for 3360690: 1
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360691%27 HTTP/1.1" 200 None
2026-04-01 06:41:26 [autobid.de] INFO: Scrape type for 3360691: 1
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360693%27 HTTP/1.1" 200 None
2026-04-01 06:41:26 [autobid.de] INFO: Scrape type for 3360693: 1
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360694%27 HTTP/1.1" 200 None
2026-04-01 06:41:26 [autobid.de] INFO: Scrape type for 3360694: 1
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360706%27 HTTP/1.1" 200 None
2026-04-01 06:41:26 [autobid.de] INFO: Scrape type for 3360706: 1
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360707%27 HTTP/1.1" 200 None
2026-04-01 06:41:26 [autobid.de] INFO: Scrape type for 3360707: 1
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361237%27 HTTP/1.1" 200 None
2026-04-01 06:41:26 [autobid.de] INFO: Scrape type for 3361237: 1
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361284%27 HTTP/1.1" 200 None
2026-04-01 06:41:26 [autobid.de] INFO: Scrape type for 3361284: 1
2026-04-01 06:41:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item caf15a99-6498-5e18-9d4b-9aeb6b691203 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33584/3358326/93371434_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:41:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 0e89bb76-52b2-5521-baee-cd1cf7345e3d due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33553/3355200/93334053_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:41:28 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-golf-plus-2-0-tdi-comfortline-3361284/details> (referer: None)
2026-04-01 06:41:28 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: b3a761e2-a224-564a-acef-770a13e610a6 with identifier: 3361284
2026-04-01 06:41:28 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b3a761e2-a224-564a-acef-770a13e610a6 with scrape type 1
2026-04-01 06:41:28 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item b3a761e2-a224-564a-acef-770a13e610a6
2026-04-01 06:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33613%252F3361284%252F93419714_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33613/3361284/93419714_hd.jpg not downloaded yet
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33613%252F3361284%252F93419704_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33613/3361284/93419704_hd.jpg not downloaded yet
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33613%252F3361284%252F93419687_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33613/3361284/93419687_hd.jpg not downloaded yet
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33613%252F3361284%252F93419713_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33613/3361284/93419713_hd.jpg not downloaded yet
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33613%252F3361284%252F93419688_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33613/3361284/93419688_hd.jpg not downloaded yet
2026-04-01 06:41:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=36> (referer: None)
2026-04-01 06:41:29 [autobid.de] INFO: Page 36/145: found 20 items
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361349%27 HTTP/1.1" 200 None
2026-04-01 06:41:29 [autobid.de] INFO: Scrape type for 3361349: 1
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361344%27 HTTP/1.1" 200 None
2026-04-01 06:41:29 [autobid.de] INFO: Scrape type for 3361344: 1
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361345%27 HTTP/1.1" 200 None
2026-04-01 06:41:29 [autobid.de] INFO: Scrape type for 3361345: 1
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361360%27 HTTP/1.1" 200 None
2026-04-01 06:41:29 [autobid.de] INFO: Scrape type for 3361360: 1
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361359%27 HTTP/1.1" 200 None
2026-04-01 06:41:29 [autobid.de] INFO: Scrape type for 3361359: 1
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361362%27 HTTP/1.1" 200 None
2026-04-01 06:41:29 [autobid.de] INFO: Scrape type for 3361362: 1
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362619%27 HTTP/1.1" 200 None
2026-04-01 06:41:29 [autobid.de] INFO: Scrape type for 3362619: 1
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362707%27 HTTP/1.1" 200 None
2026-04-01 06:41:29 [autobid.de] INFO: Scrape type for 3362707: 1
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363045%27 HTTP/1.1" 200 None
2026-04-01 06:41:29 [autobid.de] INFO: Scrape type for 3363045: 1
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363489%27 HTTP/1.1" 200 None
2026-04-01 06:41:29 [autobid.de] INFO: Scrape type for 3363489: 1
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273348753%27 HTTP/1.1" 200 None
2026-04-01 06:41:29 [autobid.de] INFO: Scrape type for 3348753: 1
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363563%27 HTTP/1.1" 200 None
2026-04-01 06:41:29 [autobid.de] INFO: Scrape type for 3363563: 1
2026-04-01 06:41:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363564%27 HTTP/1.1" 200 None
2026-04-01 06:41:30 [autobid.de] INFO: Scrape type for 3363564: 1
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363173%27 HTTP/1.1" 200 None
2026-04-01 06:41:30 [autobid.de] INFO: Scrape type for 3363173: 1
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363588%27 HTTP/1.1" 200 None
2026-04-01 06:41:30 [autobid.de] INFO: Scrape type for 3363588: 1
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363172%27 HTTP/1.1" 200 None
2026-04-01 06:41:30 [autobid.de] INFO: Scrape type for 3363172: 1
2026-04-01 06:41:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 0e89bb76-52b2-5521-baee-cd1cf7345e3d due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33553/3355200/93334052_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363727%27 HTTP/1.1" 200 None
2026-04-01 06:41:30 [autobid.de] INFO: Scrape type for 3363727: 1
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364611%27 HTTP/1.1" 200 None
2026-04-01 06:41:30 [autobid.de] INFO: Scrape type for 3364611: 1
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365082%27 HTTP/1.1" 200 None
2026-04-01 06:41:30 [autobid.de] INFO: Scrape type for 3365082: 1
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273329155%27 HTTP/1.1" 200 None
2026-04-01 06:41:30 [autobid.de] INFO: Scrape type for 3329155: 1
2026-04-01 06:41:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/skoda-fabia-1-4-ambiente-3361237/details> (referer: None)
2026-04-01 06:41:30 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 14b8d473-c79b-5a23-8049-943101fb6840 with identifier: 3361237
2026-04-01 06:41:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 14b8d473-c79b-5a23-8049-943101fb6840 with scrape type 1
2026-04-01 06:41:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 14b8d473-c79b-5a23-8049-943101fb6840
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33613%252F3361237%252F93417306_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33613/3361237/93417306_hd.jpg not downloaded yet
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33613%252F3361237%252F93417300_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33613/3361237/93417300_hd.jpg not downloaded yet
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33613%252F3361237%252F93417293_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33613/3361237/93417293_hd.jpg not downloaded yet
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33613%252F3361237%252F93417307_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33613/3361237/93417307_hd.jpg not downloaded yet
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33613%252F3361237%252F93417313_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33613/3361237/93417313_hd.jpg not downloaded yet
2026-04-01 06:41:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mercedes-benz-b-180-cdi-3363489/details> (referer: None)
2026-04-01 06:41:31 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: ffc86f7f-51dd-51e8-bc20-fd29f0726061 with identifier: 3363489
2026-04-01 06:41:31 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ffc86f7f-51dd-51e8-bc20-fd29f0726061 with scrape type 1
2026-04-01 06:41:31 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item ffc86f7f-51dd-51e8-bc20-fd29f0726061
2026-04-01 06:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33635%252F3363489%252F93501330_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33635/3363489/93501330_hd.jpg not downloaded yet
2026-04-01 06:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33635%252F3363489%252F93501329_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33635/3363489/93501329_hd.jpg not downloaded yet
2026-04-01 06:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33635%252F3363489%252F93501325_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33635/3363489/93501325_hd.jpg not downloaded yet
2026-04-01 06:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33635%252F3363489%252F93501328_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33635/3363489/93501328_hd.jpg not downloaded yet
2026-04-01 06:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33635%252F3363489%252F93501331_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:41:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33635/3363489/93501331_hd.jpg not downloaded yet
2026-04-01 06:41:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item c09fb3ae-9417-560b-8024-44d3255011e5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33590/3358989/93485613_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:41:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=37> (referer: None)
2026-04-01 06:41:34 [autobid.de] INFO: Page 37/145: found 20 items
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353320%27 HTTP/1.1" 200 None
2026-04-01 06:41:34 [autobid.de] INFO: Scrape type for 3353320: 1
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356980%27 HTTP/1.1" 200 None
2026-04-01 06:41:34 [autobid.de] INFO: Scrape type for 3356980: 1
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273329170%27 HTTP/1.1" 200 None
2026-04-01 06:41:34 [autobid.de] INFO: Scrape type for 3329170: 1
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273350580%27 HTTP/1.1" 200 None
2026-04-01 06:41:34 [autobid.de] INFO: Scrape type for 3350580: 1
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353319%27 HTTP/1.1" 200 None
2026-04-01 06:41:34 [autobid.de] INFO: Scrape type for 3353319: 1
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273315359%27 HTTP/1.1" 200 None
2026-04-01 06:41:34 [autobid.de] INFO: Scrape type for 3315359: 1
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356979%27 HTTP/1.1" 200 None
2026-04-01 06:41:34 [autobid.de] INFO: Scrape type for 3356979: 1
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273344352%27 HTTP/1.1" 200 None
2026-04-01 06:41:34 [autobid.de] INFO: Scrape type for 3344352: 1
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273350582%27 HTTP/1.1" 200 None
2026-04-01 06:41:34 [autobid.de] INFO: Scrape type for 3350582: 1
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358736%27 HTTP/1.1" 200 None
2026-04-01 06:41:34 [autobid.de] INFO: Scrape type for 3358736: 1
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358737%27 HTTP/1.1" 200 None
2026-04-01 06:41:34 [autobid.de] INFO: Scrape type for 3358737: 1
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358740%27 HTTP/1.1" 200 None
2026-04-01 06:41:34 [autobid.de] INFO: Scrape type for 3358740: 1
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361553%27 HTTP/1.1" 200 None
2026-04-01 06:41:34 [autobid.de] INFO: Scrape type for 3361553: 1
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273349611%27 HTTP/1.1" 200 None
2026-04-01 06:41:34 [autobid.de] INFO: Scrape type for 3349611: 1
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361549%27 HTTP/1.1" 200 None
2026-04-01 06:41:34 [autobid.de] INFO: Scrape type for 3361549: 1
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361551%27 HTTP/1.1" 200 None
2026-04-01 06:41:34 [autobid.de] INFO: Scrape type for 3361551: 1
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361552%27 HTTP/1.1" 200 None
2026-04-01 06:41:34 [autobid.de] INFO: Scrape type for 3361552: 1
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362631%27 HTTP/1.1" 200 None
2026-04-01 06:41:34 [autobid.de] INFO: Scrape type for 3362631: 1
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273349613%27 HTTP/1.1" 200 None
2026-04-01 06:41:34 [autobid.de] INFO: Scrape type for 3349613: 1
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:41:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361312%27 HTTP/1.1" 200 None
2026-04-01 06:41:34 [autobid.de] INFO: Scrape type for 3361312: 1
2026-04-01 06:41:42 [scrapy.extensions.logstats] INFO: Crawled 162 pages (at 9 pages/min), scraped 7 items (at 0 items/min)
2026-04-01 06:41:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 237MiB
2026-04-01 06:42:42 [scrapy.extensions.logstats] INFO: Crawled 162 pages (at 0 pages/min), scraped 7 items (at 0 items/min)
2026-04-01 06:42:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 237MiB
2026-04-01 06:43:42 [scrapy.extensions.logstats] INFO: Crawled 162 pages (at 0 pages/min), scraped 7 items (at 0 items/min)
2026-04-01 06:43:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 237MiB
2026-04-01 06:44:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 2a0cd082-6c1b-5552-a1b0-7790e999ea8e due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33591/3359001/93487131_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:44:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 2a0cd082-6c1b-5552-a1b0-7790e999ea8e due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33591/3359001/93487143_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:44:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mercedes-benz-evito-kasten-111-lang-3361312/details> (referer: None)
2026-04-01 06:44:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 84a053e5-cc2b-5832-a0b6-17c4d7bec2cf with identifier: 3361312
2026-04-01 06:44:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 84a053e5-cc2b-5832-a0b6-17c4d7bec2cf with scrape type 1
2026-04-01 06:44:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 84a053e5-cc2b-5832-a0b6-17c4d7bec2cf
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361312%252F93421640_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:44:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361312/93421640_hd.jpg not downloaded yet
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361312%252F93421647_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:44:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361312/93421647_hd.jpg not downloaded yet
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361312%252F93421631_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:44:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361312/93421631_hd.jpg not downloaded yet
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361312%252F93421633_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:44:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361312/93421633_hd.jpg not downloaded yet
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33614%252F3361312%252F93421643_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:44:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33614/3361312/93421643_hd.jpg not downloaded yet
2026-04-01 06:44:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=38> (referer: None)
2026-04-01 06:44:22 [autobid.de] INFO: Page 38/146: found 20 items
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363504%27 HTTP/1.1" 200 None
2026-04-01 06:44:22 [autobid.de] INFO: Scrape type for 3363504: 1
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361313%27 HTTP/1.1" 200 None
2026-04-01 06:44:22 [autobid.de] INFO: Scrape type for 3361313: 1
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358738%27 HTTP/1.1" 200 None
2026-04-01 06:44:22 [autobid.de] INFO: Scrape type for 3358738: 1
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358739%27 HTTP/1.1" 200 None
2026-04-01 06:44:22 [autobid.de] INFO: Scrape type for 3358739: 1
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356327%27 HTTP/1.1" 200 None
2026-04-01 06:44:22 [autobid.de] INFO: Scrape type for 3356327: 1
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273342569%27 HTTP/1.1" 200 None
2026-04-01 06:44:22 [autobid.de] INFO: Scrape type for 3342569: 1
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273342562%27 HTTP/1.1" 200 None
2026-04-01 06:44:22 [autobid.de] INFO: Scrape type for 3342562: 1
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363575%27 HTTP/1.1" 200 None
2026-04-01 06:44:22 [autobid.de] INFO: Scrape type for 3363575: 1
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273342571%27 HTTP/1.1" 200 None
2026-04-01 06:44:22 [autobid.de] INFO: Scrape type for 3342571: 1
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363577%27 HTTP/1.1" 200 None
2026-04-01 06:44:23 [autobid.de] INFO: Scrape type for 3363577: 1
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273342568%27 HTTP/1.1" 200 None
2026-04-01 06:44:23 [autobid.de] INFO: Scrape type for 3342568: 1
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273342561%27 HTTP/1.1" 200 None
2026-04-01 06:44:23 [autobid.de] INFO: Scrape type for 3342561: 1
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356328%27 HTTP/1.1" 200 None
2026-04-01 06:44:23 [autobid.de] INFO: Scrape type for 3356328: 1
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356329%27 HTTP/1.1" 200 None
2026-04-01 06:44:23 [autobid.de] INFO: Scrape type for 3356329: 1
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352583%27 HTTP/1.1" 200 None
2026-04-01 06:44:23 [autobid.de] INFO: Scrape type for 3352583: 1
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356330%27 HTTP/1.1" 200 None
2026-04-01 06:44:23 [autobid.de] INFO: Scrape type for 3356330: 1
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273351291%27 HTTP/1.1" 200 None
2026-04-01 06:44:23 [autobid.de] INFO: Scrape type for 3351291: 1
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352582%27 HTTP/1.1" 200 None
2026-04-01 06:44:23 [autobid.de] INFO: Scrape type for 3352582: 1
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273351292%27 HTTP/1.1" 200 None
2026-04-01 06:44:23 [autobid.de] INFO: Scrape type for 3351292: 1
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365123%27 HTTP/1.1" 200 None
2026-04-01 06:44:23 [autobid.de] INFO: Scrape type for 3365123: 1
2026-04-01 06:44:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-caddy-kasten-2-0-tdi-bmt-dsg-3349613/details> (referer: None)
2026-04-01 06:44:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: e0e32956-d599-5102-ab78-5e0eb729fd45 with identifier: 3349613
2026-04-01 06:44:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e0e32956-d599-5102-ab78-5e0eb729fd45 with scrape type 1
2026-04-01 06:44:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item e0e32956-d599-5102-ab78-5e0eb729fd45
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33497%252F3349613%252F93302870_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:44:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33497/3349613/93302870_hd.jpg not downloaded yet
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33497%252F3349613%252F93302873_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:44:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33497/3349613/93302873_hd.jpg not downloaded yet
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33497%252F3349613%252F93302849_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:44:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33497/3349613/93302849_hd.jpg not downloaded yet
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33497%252F3349613%252F93302851_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:44:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33497/3349613/93302851_hd.jpg not downloaded yet
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33497%252F3349613%252F93302853_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:44:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33497/3349613/93302853_hd.jpg not downloaded yet
2026-04-01 06:44:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 2a0cd082-6c1b-5552-a1b0-7790e999ea8e due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33591/3359001/93487147_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:44:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 2a0cd082-6c1b-5552-a1b0-7790e999ea8e due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33591/3359001/93487139_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:44:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=39> (referer: None)
2026-04-01 06:44:27 [autobid.de] INFO: Page 39/145: found 20 items
2026-04-01 06:44:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365560%27 HTTP/1.1" 200 None
2026-04-01 06:44:27 [autobid.de] INFO: Scrape type for 3365560: 1
2026-04-01 06:44:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273335118%27 HTTP/1.1" 200 None
2026-04-01 06:44:27 [autobid.de] INFO: Scrape type for 3335118: 1
2026-04-01 06:44:27 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 2a0cd082-6c1b-5552-a1b0-7790e999ea8e with identifier: 3359001
2026-04-01 06:44:27 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 06:44:27 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=green+Metallic HTTP/1.1" 200 None
2026-04-01 06:44:27 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 06:44:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 06:44:27 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 2a0cd082-6c1b-5552-a1b0-7790e999ea8e with identifier: 3359001
2026-04-01 06:44:27 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: 2a0cd082-6c1b-5552-a1b0-7790e999ea8e with identifier: 3359001 to the API
2026-04-01 06:44:27 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 06:44:27 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 06:44:28 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33591/3359001/93487134_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 81247',
'body_type': 'Saloon',
'category': 'Saloon',
'color': 'green Metallic',
'currency': 'EUR',
'emission_standard': 'EURO 4',
'engine': '1.3L',
'engine_horse_power': 60,
'expiration_date': '2026-04-01T12:00:00Z',
'fuel_type': 'Petrol',
'id': '2a0cd082-6c1b-5552-a1b0-7790e999ea8e',
'is_damaged': False,
'is_operable': True,
'is_vat_deductible': False,
'link_to_web_offer': 'https://autobid.de/en/item/ford-fiesta-1-3-ambiente-3359001',
'listing': 'auction',
'location': 'Wilhelmshaven',
'make': 'Ford',
'model': 'Fiesta',
'odometer': 252400,
'odometer_reading_unit': 'km',
'origin_country_code': 'DE',
'original_photo_count': 5,
'photo_urls': ['https://cdn.autobid.de/data/cars/33591/3359001/93487134_hd.jpg',
'https://cdn.autobid.de/data/cars/33591/3359001/93487139_hd.jpg',
'https://cdn.autobid.de/data/cars/33591/3359001/93487147_hd.jpg',
'https://cdn.autobid.de/data/cars/33591/3359001/93487143_hd.jpg',
'https://cdn.autobid.de/data/cars/33591/3359001/93487131_hd.jpg'],
'photos': [],
'price': 800,
'price_includes_vat': False,
'registration_date': '2006-08-01',
'seats_number': 5,
'seller_name': None,
'title': 'Ford Fiesta 1,3 "Ambiente"',
'transmission': 'manual',
'vin': None,
'year': 2006}
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359001%27 HTTP/1.1" 200 None
2026-04-01 06:44:28 [autobid.de] INFO: Saving data for 3359001: {'created_time': 1775025868.072748, 'last_price_update_time': 1775025868.072761, 'auction_closing_time': 1775044800.0}
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3359001') HTTP/1.1" 204 0
2026-04-01 06:44:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 2a0cd082-6c1b-5552-a1b0-7790e999ea8e due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33591/3359001/93487134_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352581%27 HTTP/1.1" 200 None
2026-04-01 06:44:28 [autobid.de] INFO: Scrape type for 3352581: 1
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359083%27 HTTP/1.1" 200 None
2026-04-01 06:44:28 [autobid.de] INFO: Scrape type for 3359083: 1
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366843%27 HTTP/1.1" 200 None
2026-04-01 06:44:28 [autobid.de] INFO: Scrape type for 3366843: 1
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353626%27 HTTP/1.1" 200 None
2026-04-01 06:44:28 [autobid.de] INFO: Scrape type for 3353626: 1
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355217%27 HTTP/1.1" 200 None
2026-04-01 06:44:28 [autobid.de] INFO: Scrape type for 3355217: 1
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361368%27 HTTP/1.1" 200 None
2026-04-01 06:44:28 [autobid.de] INFO: Scrape type for 3361368: 1
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273349475%27 HTTP/1.1" 200 None
2026-04-01 06:44:28 [autobid.de] INFO: Scrape type for 3349475: 1
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358149%27 HTTP/1.1" 200 None
2026-04-01 06:44:28 [autobid.de] INFO: Scrape type for 3358149: 1
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273351635%27 HTTP/1.1" 200 None
2026-04-01 06:44:28 [autobid.de] INFO: Scrape type for 3351635: 1
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360004%27 HTTP/1.1" 200 None
2026-04-01 06:44:28 [autobid.de] INFO: Scrape type for 3360004: 1
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361364%27 HTTP/1.1" 200 None
2026-04-01 06:44:28 [autobid.de] INFO: Scrape type for 3361364: 1
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359718%27 HTTP/1.1" 200 None
2026-04-01 06:44:28 [autobid.de] INFO: Scrape type for 3359718: 1
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358154%27 HTTP/1.1" 200 None
2026-04-01 06:44:28 [autobid.de] INFO: Scrape type for 3358154: 1
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361346%27 HTTP/1.1" 200 None
2026-04-01 06:44:28 [autobid.de] INFO: Scrape type for 3361346: 1
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360567%27 HTTP/1.1" 200 None
2026-04-01 06:44:28 [autobid.de] INFO: Scrape type for 3360567: 1
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361972%27 HTTP/1.1" 200 None
2026-04-01 06:44:28 [autobid.de] INFO: Scrape type for 3361972: 1
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358152%27 HTTP/1.1" 200 None
2026-04-01 06:44:28 [autobid.de] INFO: Scrape type for 3358152: 1
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358148%27 HTTP/1.1" 200 None
2026-04-01 06:44:28 [autobid.de] INFO: Scrape type for 3358148: 1
2026-04-01 06:44:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item c09fb3ae-9417-560b-8024-44d3255011e5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33590/3358989/93485615_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:44:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item c09fb3ae-9417-560b-8024-44d3255011e5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33590/3358989/93485619_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:44:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/hyundai-i30-1-4-fifa-world-cup-edition-3358148/details> (referer: None)
2026-04-01 06:44:30 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 6f9f5d2d-b377-5cf3-9234-359ced998497 with identifier: 3358148
2026-04-01 06:44:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6f9f5d2d-b377-5cf3-9234-359ced998497 with scrape type 1
2026-04-01 06:44:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 6f9f5d2d-b377-5cf3-9234-359ced998497
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33582%252F3358148%252F93389372_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:44:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33582/3358148/93389372_hd.jpg not downloaded yet
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33582%252F3358148%252F93389373_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:44:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33582/3358148/93389373_hd.jpg not downloaded yet
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33582%252F3358148%252F93389374_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:44:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33582/3358148/93389374_hd.jpg not downloaded yet
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33582%252F3358148%252F93389375_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:44:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33582/3358148/93389375_hd.jpg not downloaded yet
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33582%252F3358148%252F93389376_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:44:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33582/3358148/93389376_hd.jpg not downloaded yet
2026-04-01 06:44:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=40> (referer: None)
2026-04-01 06:44:30 [autobid.de] INFO: Page 40/145: found 20 items
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361320%27 HTTP/1.1" 200 None
2026-04-01 06:44:30 [autobid.de] INFO: Scrape type for 3361320: 1
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360568%27 HTTP/1.1" 200 None
2026-04-01 06:44:30 [autobid.de] INFO: Scrape type for 3360568: 1
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359753%27 HTTP/1.1" 200 None
2026-04-01 06:44:30 [autobid.de] INFO: Scrape type for 3359753: 1
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273349457%27 HTTP/1.1" 200 None
2026-04-01 06:44:30 [autobid.de] INFO: Scrape type for 3349457: 1
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358150%27 HTTP/1.1" 200 None
2026-04-01 06:44:30 [autobid.de] INFO: Scrape type for 3358150: 1
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362025%27 HTTP/1.1" 200 None
2026-04-01 06:44:30 [autobid.de] INFO: Scrape type for 3362025: 1
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361427%27 HTTP/1.1" 200 None
2026-04-01 06:44:30 [autobid.de] INFO: Scrape type for 3361427: 1
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354122%27 HTTP/1.1" 200 None
2026-04-01 06:44:30 [autobid.de] INFO: Scrape type for 3354122: 1
2026-04-01 06:44:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357522%27 HTTP/1.1" 200 None
2026-04-01 06:44:31 [autobid.de] INFO: Scrape type for 3357522: 1
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358153%27 HTTP/1.1" 200 None
2026-04-01 06:44:31 [autobid.de] INFO: Scrape type for 3358153: 1
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362026%27 HTTP/1.1" 200 None
2026-04-01 06:44:31 [autobid.de] INFO: Scrape type for 3362026: 1
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361347%27 HTTP/1.1" 200 None
2026-04-01 06:44:31 [autobid.de] INFO: Scrape type for 3361347: 1
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361321%27 HTTP/1.1" 200 None
2026-04-01 06:44:31 [autobid.de] INFO: Scrape type for 3361321: 1
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358297%27 HTTP/1.1" 200 None
2026-04-01 06:44:31 [autobid.de] INFO: Scrape type for 3358297: 1
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273349704%27 HTTP/1.1" 200 None
2026-04-01 06:44:31 [autobid.de] INFO: Scrape type for 3349704: 1
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358335%27 HTTP/1.1" 200 None
2026-04-01 06:44:31 [autobid.de] INFO: Scrape type for 3358335: 1
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355226%27 HTTP/1.1" 200 None
2026-04-01 06:44:31 [autobid.de] INFO: Scrape type for 3355226: 1
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361426%27 HTTP/1.1" 200 None
2026-04-01 06:44:31 [autobid.de] INFO: Scrape type for 3361426: 1
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353345%27 HTTP/1.1" 200 None
2026-04-01 06:44:31 [autobid.de] INFO: Scrape type for 3353345: 1
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273349706%27 HTTP/1.1" 200 None
2026-04-01 06:44:31 [autobid.de] INFO: Scrape type for 3349706: 1
2026-04-01 06:44:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/bmw-x1-xdrive20i-steptronic-advantage-3358152/details> (referer: None)
2026-04-01 06:44:31 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 19e8bd11-a744-5a9a-aae9-151cb9ff087b with identifier: 3358152
2026-04-01 06:44:31 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 19e8bd11-a744-5a9a-aae9-151cb9ff087b with scrape type 1
2026-04-01 06:44:31 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 19e8bd11-a744-5a9a-aae9-151cb9ff087b
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33582%252F3358152%252F93386195_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:44:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33582/3358152/93386195_hd.jpg not downloaded yet
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33582%252F3358152%252F93386196_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:44:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33582/3358152/93386196_hd.jpg not downloaded yet
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33582%252F3358152%252F93386197_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:44:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33582/3358152/93386197_hd.jpg not downloaded yet
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33582%252F3358152%252F93386198_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:44:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33582/3358152/93386198_hd.jpg not downloaded yet
2026-04-01 06:44:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33582%252F3358152%252F93386199_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:44:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33582/3358152/93386199_hd.jpg not downloaded yet
2026-04-01 06:44:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item c09fb3ae-9417-560b-8024-44d3255011e5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33590/3358989/93485611_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:44:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=41> (referer: None)
2026-04-01 06:44:34 [autobid.de] INFO: Page 41/145: found 20 items
2026-04-01 06:44:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362802%27 HTTP/1.1" 200 None
2026-04-01 06:44:34 [autobid.de] INFO: Scrape type for 3362802: 1
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361413%27 HTTP/1.1" 200 None
2026-04-01 06:44:35 [autobid.de] INFO: Scrape type for 3361413: 1
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361332%27 HTTP/1.1" 200 None
2026-04-01 06:44:35 [autobid.de] INFO: Scrape type for 3361332: 1
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353654%27 HTTP/1.1" 200 None
2026-04-01 06:44:35 [autobid.de] INFO: Scrape type for 3353654: 1
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358147%27 HTTP/1.1" 200 None
2026-04-01 06:44:35 [autobid.de] INFO: Scrape type for 3358147: 1
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357026%27 HTTP/1.1" 200 None
2026-04-01 06:44:35 [autobid.de] INFO: Scrape type for 3357026: 1
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360572%27 HTTP/1.1" 200 None
2026-04-01 06:44:35 [autobid.de] INFO: Scrape type for 3360572: 1
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359749%27 HTTP/1.1" 200 None
2026-04-01 06:44:35 [autobid.de] INFO: Scrape type for 3359749: 1
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360041%27 HTTP/1.1" 200 None
2026-04-01 06:44:35 [autobid.de] INFO: Scrape type for 3360041: 1
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358241%27 HTTP/1.1" 200 None
2026-04-01 06:44:35 [autobid.de] INFO: Scrape type for 3358241: 1
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358165%27 HTTP/1.1" 200 None
2026-04-01 06:44:35 [autobid.de] INFO: Scrape type for 3358165: 1
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352395%27 HTTP/1.1" 200 None
2026-04-01 06:44:35 [autobid.de] INFO: Scrape type for 3352395: 1
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360005%27 HTTP/1.1" 200 None
2026-04-01 06:44:35 [autobid.de] INFO: Scrape type for 3360005: 1
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361363%27 HTTP/1.1" 200 None
2026-04-01 06:44:35 [autobid.de] INFO: Scrape type for 3361363: 1
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361337%27 HTTP/1.1" 200 None
2026-04-01 06:44:35 [autobid.de] INFO: Scrape type for 3361337: 1
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363230%27 HTTP/1.1" 200 None
2026-04-01 06:44:35 [autobid.de] INFO: Scrape type for 3363230: 1
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360576%27 HTTP/1.1" 200 None
2026-04-01 06:44:35 [autobid.de] INFO: Scrape type for 3360576: 1
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356482%27 HTTP/1.1" 200 None
2026-04-01 06:44:35 [autobid.de] INFO: Scrape type for 3356482: 1
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357392%27 HTTP/1.1" 200 None
2026-04-01 06:44:35 [autobid.de] INFO: Scrape type for 3357392: 1
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:44:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358137%27 HTTP/1.1" 200 None
2026-04-01 06:44:35 [autobid.de] INFO: Scrape type for 3358137: 1
2026-04-01 06:44:42 [scrapy.extensions.logstats] INFO: Crawled 170 pages (at 8 pages/min), scraped 8 items (at 1 items/min)
2026-04-01 06:44:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 239MiB
2026-04-01 06:45:42 [scrapy.extensions.logstats] INFO: Crawled 170 pages (at 0 pages/min), scraped 8 items (at 0 items/min)
2026-04-01 06:45:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 239MiB
2026-04-01 06:46:42 [scrapy.extensions.logstats] INFO: Crawled 170 pages (at 0 pages/min), scraped 8 items (at 0 items/min)
2026-04-01 06:46:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 239MiB
2026-04-01 06:47:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item f92d4fd0-f2f4-5167-8590-8960c79d4dbb due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33591/3359034/93506308_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:47:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 3b9e2237-3016-5051-948a-3584005cad82 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33591/3359033/93491017_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:47:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/kia-picanto-1-2-automatik-gt-line-3358137/details> (referer: None)
2026-04-01 06:47:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 502bcdde-4897-5a33-b3f3-b41bb977e2de with identifier: 3358137
2026-04-01 06:47:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 502bcdde-4897-5a33-b3f3-b41bb977e2de with scrape type 1
2026-04-01 06:47:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 502bcdde-4897-5a33-b3f3-b41bb977e2de
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33582%252F3358137%252F93453223_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:47:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33582/3358137/93453223_hd.jpg not downloaded yet
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33582%252F3358137%252F93453224_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:47:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33582/3358137/93453224_hd.jpg not downloaded yet
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33582%252F3358137%252F93453225_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:47:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33582/3358137/93453225_hd.jpg not downloaded yet
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33582%252F3358137%252F93453226_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:47:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33582/3358137/93453226_hd.jpg not downloaded yet
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33582%252F3358137%252F93453227_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:47:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33582/3358137/93453227_hd.jpg not downloaded yet
2026-04-01 06:47:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=42> (referer: None)
2026-04-01 06:47:22 [autobid.de] INFO: Page 42/145: found 20 items
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361323%27 HTTP/1.1" 200 None
2026-04-01 06:47:22 [autobid.de] INFO: Scrape type for 3361323: 1
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361336%27 HTTP/1.1" 200 None
2026-04-01 06:47:22 [autobid.de] INFO: Scrape type for 3361336: 1
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361022%27 HTTP/1.1" 200 None
2026-04-01 06:47:22 [autobid.de] INFO: Scrape type for 3361022: 1
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361825%27 HTTP/1.1" 200 None
2026-04-01 06:47:22 [autobid.de] INFO: Scrape type for 3361825: 1
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360569%27 HTTP/1.1" 200 None
2026-04-01 06:47:22 [autobid.de] INFO: Scrape type for 3360569: 1
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361338%27 HTTP/1.1" 200 None
2026-04-01 06:47:22 [autobid.de] INFO: Scrape type for 3361338: 1
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361369%27 HTTP/1.1" 200 None
2026-04-01 06:47:22 [autobid.de] INFO: Scrape type for 3361369: 1
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359750%27 HTTP/1.1" 200 None
2026-04-01 06:47:22 [autobid.de] INFO: Scrape type for 3359750: 1
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360570%27 HTTP/1.1" 200 None
2026-04-01 06:47:22 [autobid.de] INFO: Scrape type for 3360570: 1
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361817%27 HTTP/1.1" 200 None
2026-04-01 06:47:22 [autobid.de] INFO: Scrape type for 3361817: 1
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361348%27 HTTP/1.1" 200 None
2026-04-01 06:47:22 [autobid.de] INFO: Scrape type for 3361348: 1
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359748%27 HTTP/1.1" 200 None
2026-04-01 06:47:22 [autobid.de] INFO: Scrape type for 3359748: 1
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360575%27 HTTP/1.1" 200 None
2026-04-01 06:47:22 [autobid.de] INFO: Scrape type for 3360575: 1
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361414%27 HTTP/1.1" 200 None
2026-04-01 06:47:22 [autobid.de] INFO: Scrape type for 3361414: 1
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361333%27 HTTP/1.1" 200 None
2026-04-01 06:47:22 [autobid.de] INFO: Scrape type for 3361333: 1
2026-04-01 06:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361365%27 HTTP/1.1" 200 None
2026-04-01 06:47:23 [autobid.de] INFO: Scrape type for 3361365: 1
2026-04-01 06:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359747%27 HTTP/1.1" 200 None
2026-04-01 06:47:23 [autobid.de] INFO: Scrape type for 3359747: 1
2026-04-01 06:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361367%27 HTTP/1.1" 200 None
2026-04-01 06:47:23 [autobid.de] INFO: Scrape type for 3361367: 1
2026-04-01 06:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361334%27 HTTP/1.1" 200 None
2026-04-01 06:47:23 [autobid.de] INFO: Scrape type for 3361334: 1
2026-04-01 06:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359752%27 HTTP/1.1" 200 None
2026-04-01 06:47:23 [autobid.de] INFO: Scrape type for 3359752: 1
2026-04-01 06:47:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-t6-multivan-2-0-tdi-4motion-generation-six-3357392/details> (referer: None)
2026-04-01 06:47:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 90f4c135-3398-57cf-be61-7a8401217c1b with identifier: 3357392
2026-04-01 06:47:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 90f4c135-3398-57cf-be61-7a8401217c1b with scrape type 1
2026-04-01 06:47:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 90f4c135-3398-57cf-be61-7a8401217c1b
2026-04-01 06:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33574%252F3357392%252F93425239_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:47:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33574/3357392/93425239_hd.jpg not downloaded yet
2026-04-01 06:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33574%252F3357392%252F93425240_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:47:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33574/3357392/93425240_hd.jpg not downloaded yet
2026-04-01 06:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33574%252F3357392%252F93425241_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:47:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33574/3357392/93425241_hd.jpg not downloaded yet
2026-04-01 06:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33574%252F3357392%252F93425242_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:47:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33574/3357392/93425242_hd.jpg not downloaded yet
2026-04-01 06:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33574%252F3357392%252F93425243_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:47:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33574/3357392/93425243_hd.jpg not downloaded yet
2026-04-01 06:47:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 3b9e2237-3016-5051-948a-3584005cad82 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33591/3359033/93491019_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:47:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 3b9e2237-3016-5051-948a-3584005cad82 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33591/3359033/93491022_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:47:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=43> (referer: None)
2026-04-01 06:47:26 [autobid.de] INFO: Page 43/145: found 20 items
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360574%27 HTTP/1.1" 200 None
2026-04-01 06:47:26 [autobid.de] INFO: Scrape type for 3360574: 1
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360571%27 HTTP/1.1" 200 None
2026-04-01 06:47:26 [autobid.de] INFO: Scrape type for 3360571: 1
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360573%27 HTTP/1.1" 200 None
2026-04-01 06:47:26 [autobid.de] INFO: Scrape type for 3360573: 1
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361675%27 HTTP/1.1" 200 None
2026-04-01 06:47:26 [autobid.de] INFO: Scrape type for 3361675: 1
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365497%27 HTTP/1.1" 200 None
2026-04-01 06:47:26 [autobid.de] INFO: Scrape type for 3365497: 1
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364355%27 HTTP/1.1" 200 None
2026-04-01 06:47:26 [autobid.de] INFO: Scrape type for 3364355: 1
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359999%27 HTTP/1.1" 200 None
2026-04-01 06:47:26 [autobid.de] INFO: Scrape type for 3359999: 1
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364299%27 HTTP/1.1" 200 None
2026-04-01 06:47:26 [autobid.de] INFO: Scrape type for 3364299: 1
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364360%27 HTTP/1.1" 200 None
2026-04-01 06:47:26 [autobid.de] INFO: Scrape type for 3364360: 1
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365496%27 HTTP/1.1" 200 None
2026-04-01 06:47:26 [autobid.de] INFO: Scrape type for 3365496: 1
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362884%27 HTTP/1.1" 200 None
2026-04-01 06:47:26 [autobid.de] INFO: Scrape type for 3362884: 1
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357547%27 HTTP/1.1" 200 None
2026-04-01 06:47:26 [autobid.de] INFO: Scrape type for 3357547: 1
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364356%27 HTTP/1.1" 200 None
2026-04-01 06:47:26 [autobid.de] INFO: Scrape type for 3364356: 1
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365362%27 HTTP/1.1" 200 None
2026-04-01 06:47:26 [autobid.de] INFO: Scrape type for 3365362: 1
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365526%27 HTTP/1.1" 200 None
2026-04-01 06:47:26 [autobid.de] INFO: Scrape type for 3365526: 1
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364300%27 HTTP/1.1" 200 None
2026-04-01 06:47:26 [autobid.de] INFO: Scrape type for 3364300: 1
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362803%27 HTTP/1.1" 200 None
2026-04-01 06:47:26 [autobid.de] INFO: Scrape type for 3362803: 1
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361548%27 HTTP/1.1" 200 None
2026-04-01 06:47:26 [autobid.de] INFO: Scrape type for 3361548: 1
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358276%27 HTTP/1.1" 200 None
2026-04-01 06:47:26 [autobid.de] INFO: Scrape type for 3358276: 1
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362656%27 HTTP/1.1" 200 None
2026-04-01 06:47:26 [autobid.de] INFO: Scrape type for 3362656: 1
2026-04-01 06:47:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item b3a761e2-a224-564a-acef-770a13e610a6 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33613/3361284/93419688_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:47:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 14b8d473-c79b-5a23-8049-943101fb6840 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33613/3361237/93417313_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:47:28 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/subaru-solterra-comfort-3362656/details> (referer: None)
2026-04-01 06:47:28 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 31416781-87c9-51f7-b738-094cdc420be1 with identifier: 3362656
2026-04-01 06:47:28 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 31416781-87c9-51f7-b738-094cdc420be1 with scrape type 1
2026-04-01 06:47:28 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 31416781-87c9-51f7-b738-094cdc420be1
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362656%252F93530323_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362656/93530323_hd.jpg not downloaded yet
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362656%252F93530325_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362656/93530325_hd.jpg not downloaded yet
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362656%252F93530321_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362656/93530321_hd.jpg not downloaded yet
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362656%252F93530318_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362656/93530318_hd.jpg not downloaded yet
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362656%252F93530326_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362656/93530326_hd.jpg not downloaded yet
2026-04-01 06:47:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=44> (referer: None)
2026-04-01 06:47:29 [autobid.de] INFO: Page 44/145: found 20 items
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273283914%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [autobid.de] INFO: Scrape type for 3283914: 1
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355814%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [autobid.de] INFO: Scrape type for 3355814: 1
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358300%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [autobid.de] INFO: Scrape type for 3358300: 1
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364357%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [autobid.de] INFO: Scrape type for 3364357: 1
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358287%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [autobid.de] INFO: Scrape type for 3358287: 1
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358270%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [autobid.de] INFO: Scrape type for 3358270: 1
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365104%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [autobid.de] INFO: Scrape type for 3365104: 1
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273351258%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [autobid.de] INFO: Scrape type for 3351258: 1
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358271%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [autobid.de] INFO: Scrape type for 3358271: 1
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358278%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [autobid.de] INFO: Scrape type for 3358278: 1
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363670%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [autobid.de] INFO: Scrape type for 3363670: 1
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347465%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [autobid.de] INFO: Scrape type for 3347465: 1
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361908%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [autobid.de] INFO: Scrape type for 3361908: 1
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347491%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [autobid.de] INFO: Scrape type for 3347491: 1
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364361%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [autobid.de] INFO: Scrape type for 3364361: 1
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347492%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [autobid.de] INFO: Scrape type for 3347492: 1
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362919%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [autobid.de] INFO: Scrape type for 3362919: 1
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347493%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [autobid.de] INFO: Scrape type for 3347493: 1
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361750%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [autobid.de] INFO: Scrape type for 3361750: 1
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347495%27 HTTP/1.1" 200 None
2026-04-01 06:47:29 [autobid.de] INFO: Scrape type for 3347495: 1
2026-04-01 06:47:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/volvo-c40-recharge-pure-electric-2wd-ultimate-3358276/details> (referer: None)
2026-04-01 06:47:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item ffc86f7f-51dd-51e8-bc20-fd29f0726061 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33635/3363489/93501331_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:47:30 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 7d5021fd-a243-5632-b20a-e47e311152f8 with identifier: 3358276
2026-04-01 06:47:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 7d5021fd-a243-5632-b20a-e47e311152f8 with scrape type 1
2026-04-01 06:47:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 7d5021fd-a243-5632-b20a-e47e311152f8
2026-04-01 06:47:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33583%252F3358276%252F93488074_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:47:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33583/3358276/93488074_hd.jpg not downloaded yet
2026-04-01 06:47:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33583%252F3358276%252F93488075_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:47:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33583/3358276/93488075_hd.jpg not downloaded yet
2026-04-01 06:47:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33583%252F3358276%252F93488076_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:47:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33583/3358276/93488076_hd.jpg not downloaded yet
2026-04-01 06:47:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33583%252F3358276%252F93488077_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:47:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33583/3358276/93488077_hd.jpg not downloaded yet
2026-04-01 06:47:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33583%252F3358276%252F93488078_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:47:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33583/3358276/93488078_hd.jpg not downloaded yet
2026-04-01 06:47:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/rader-reifen-paket-10-satz-3347495/details> (referer: None)
2026-04-01 06:47:31 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 94fcf76e-1b18-53ae-a62f-86c68c9376cd with identifier: 3347495
2026-04-01 06:47:31 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 94fcf76e-1b18-53ae-a62f-86c68c9376cd with scrape type 1
2026-04-01 06:47:31 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 94fcf76e-1b18-53ae-a62f-86c68c9376cd
2026-04-01 06:47:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33475%252F3347495%252F93143265_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:47:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33475/3347495/93143265_hd.jpg not downloaded yet
2026-04-01 06:47:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item ffc86f7f-51dd-51e8-bc20-fd29f0726061 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33635/3363489/93501328_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:47:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=45> (referer: None)
2026-04-01 06:47:35 [autobid.de] INFO: Page 45/145: found 20 items
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362921%27 HTTP/1.1" 200 None
2026-04-01 06:47:35 [autobid.de] INFO: Scrape type for 3362921: 1
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347494%27 HTTP/1.1" 200 None
2026-04-01 06:47:35 [autobid.de] INFO: Scrape type for 3347494: 1
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361838%27 HTTP/1.1" 200 None
2026-04-01 06:47:35 [autobid.de] INFO: Scrape type for 3361838: 1
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347496%27 HTTP/1.1" 200 None
2026-04-01 06:47:35 [autobid.de] INFO: Scrape type for 3347496: 1
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364253%27 HTTP/1.1" 200 None
2026-04-01 06:47:35 [autobid.de] INFO: Scrape type for 3364253: 1
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347497%27 HTTP/1.1" 200 None
2026-04-01 06:47:35 [autobid.de] INFO: Scrape type for 3347497: 1
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363265%27 HTTP/1.1" 200 None
2026-04-01 06:47:35 [autobid.de] INFO: Scrape type for 3363265: 1
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347498%27 HTTP/1.1" 200 None
2026-04-01 06:47:35 [autobid.de] INFO: Scrape type for 3347498: 1
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362898%27 HTTP/1.1" 200 None
2026-04-01 06:47:35 [autobid.de] INFO: Scrape type for 3362898: 1
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347456%27 HTTP/1.1" 200 None
2026-04-01 06:47:35 [autobid.de] INFO: Scrape type for 3347456: 1
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364364%27 HTTP/1.1" 200 None
2026-04-01 06:47:35 [autobid.de] INFO: Scrape type for 3364364: 1
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347499%27 HTTP/1.1" 200 None
2026-04-01 06:47:35 [autobid.de] INFO: Scrape type for 3347499: 1
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363292%27 HTTP/1.1" 200 None
2026-04-01 06:47:35 [autobid.de] INFO: Scrape type for 3363292: 1
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347501%27 HTTP/1.1" 200 None
2026-04-01 06:47:35 [autobid.de] INFO: Scrape type for 3347501: 1
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364238%27 HTTP/1.1" 200 None
2026-04-01 06:47:35 [autobid.de] INFO: Scrape type for 3364238: 1
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347504%27 HTTP/1.1" 200 None
2026-04-01 06:47:35 [autobid.de] INFO: Scrape type for 3347504: 1
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362820%27 HTTP/1.1" 200 None
2026-04-01 06:47:35 [autobid.de] INFO: Scrape type for 3362820: 1
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347506%27 HTTP/1.1" 200 None
2026-04-01 06:47:35 [autobid.de] INFO: Scrape type for 3347506: 1
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361826%27 HTTP/1.1" 200 None
2026-04-01 06:47:35 [autobid.de] INFO: Scrape type for 3361826: 1
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:47:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347507%27 HTTP/1.1" 200 None
2026-04-01 06:47:35 [autobid.de] INFO: Scrape type for 3347507: 1
2026-04-01 06:47:42 [scrapy.extensions.logstats] INFO: Crawled 179 pages (at 9 pages/min), scraped 8 items (at 0 items/min)
2026-04-01 06:47:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 239MiB
2026-04-01 06:48:42 [scrapy.extensions.logstats] INFO: Crawled 179 pages (at 0 pages/min), scraped 8 items (at 0 items/min)
2026-04-01 06:48:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 239MiB
2026-04-01 06:49:42 [scrapy.extensions.logstats] INFO: Crawled 179 pages (at 0 pages/min), scraped 8 items (at 0 items/min)
2026-04-01 06:49:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 239MiB
2026-04-01 06:50:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 84a053e5-cc2b-5832-a0b6-17c4d7bec2cf due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33614/3361312/93421643_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:50:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item e0e32956-d599-5102-ab78-5e0eb729fd45 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33497/3349613/93302853_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:50:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/rader-reifen-paket-10-satze-3347507/details> (referer: None)
2026-04-01 06:50:22 [crawlers.middlewares.id_gen_middleware] INFO: Duplicate ID generated for item: 94fcf76e-1b18-53ae-a62f-86c68c9376cd with identifier: 3347507, dropping item
2026-04-01 06:50:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=46> (referer: None)
2026-04-01 06:50:22 [autobid.de] INFO: Page 46/145: found 20 items
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347510%27 HTTP/1.1" 200 None
2026-04-01 06:50:22 [autobid.de] INFO: Scrape type for 3347510: 1
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347511%27 HTTP/1.1" 200 None
2026-04-01 06:50:22 [autobid.de] INFO: Scrape type for 3347511: 1
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347512%27 HTTP/1.1" 200 None
2026-04-01 06:50:22 [autobid.de] INFO: Scrape type for 3347512: 1
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347513%27 HTTP/1.1" 200 None
2026-04-01 06:50:22 [autobid.de] INFO: Scrape type for 3347513: 1
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347514%27 HTTP/1.1" 200 None
2026-04-01 06:50:22 [autobid.de] INFO: Scrape type for 3347514: 1
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347528%27 HTTP/1.1" 200 None
2026-04-01 06:50:22 [autobid.de] INFO: Scrape type for 3347528: 1
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347562%27 HTTP/1.1" 200 None
2026-04-01 06:50:22 [autobid.de] INFO: Scrape type for 3347562: 1
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347564%27 HTTP/1.1" 200 None
2026-04-01 06:50:22 [autobid.de] INFO: Scrape type for 3347564: 1
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357128%27 HTTP/1.1" 200 None
2026-04-01 06:50:22 [autobid.de] INFO: Scrape type for 3357128: 1
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359369%27 HTTP/1.1" 200 None
2026-04-01 06:50:22 [autobid.de] INFO: Scrape type for 3359369: 1
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359370%27 HTTP/1.1" 200 None
2026-04-01 06:50:22 [autobid.de] INFO: Scrape type for 3359370: 1
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361957%27 HTTP/1.1" 200 None
2026-04-01 06:50:22 [autobid.de] INFO: Scrape type for 3361957: 1
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361967%27 HTTP/1.1" 200 None
2026-04-01 06:50:22 [autobid.de] INFO: Scrape type for 3361967: 1
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361958%27 HTTP/1.1" 200 None
2026-04-01 06:50:22 [autobid.de] INFO: Scrape type for 3361958: 1
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361968%27 HTTP/1.1" 200 None
2026-04-01 06:50:22 [autobid.de] INFO: Scrape type for 3361968: 1
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361959%27 HTTP/1.1" 200 None
2026-04-01 06:50:22 [autobid.de] INFO: Scrape type for 3361959: 1
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361966%27 HTTP/1.1" 200 None
2026-04-01 06:50:22 [autobid.de] INFO: Scrape type for 3361966: 1
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361960%27 HTTP/1.1" 200 None
2026-04-01 06:50:22 [autobid.de] INFO: Scrape type for 3361960: 1
2026-04-01 06:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361965%27 HTTP/1.1" 200 None
2026-04-01 06:50:23 [autobid.de] INFO: Scrape type for 3361965: 1
2026-04-01 06:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361961%27 HTTP/1.1" 200 None
2026-04-01 06:50:23 [autobid.de] INFO: Scrape type for 3361961: 1
2026-04-01 06:50:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/bmw-x5-30-d-xdrive-m-sport-connected-professional-uvp-127-490-00-3361826/details> (referer: None)
2026-04-01 06:50:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 14193e74-947e-5162-9770-5016e30f258b with identifier: 3361826
2026-04-01 06:50:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 14193e74-947e-5162-9770-5016e30f258b with scrape type 1
2026-04-01 06:50:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 3 photos for item 14193e74-947e-5162-9770-5016e30f258b
2026-04-01 06:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33619%252F3361826%252F93455541_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:50:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33619/3361826/93455541_hd.jpg not downloaded yet
2026-04-01 06:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33619%252F3361826%252F93455542_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:50:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33619/3361826/93455542_hd.jpg not downloaded yet
2026-04-01 06:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33619%252F3361826%252F93455543_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:50:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33619/3361826/93455543_hd.jpg not downloaded yet
2026-04-01 06:50:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item e0e32956-d599-5102-ab78-5e0eb729fd45 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33497/3349613/93302851_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:50:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/rader-reifen-paket-10-satz-3347506/details> (referer: None)
2026-04-01 06:50:24 [crawlers.middlewares.id_gen_middleware] INFO: Duplicate ID generated for item: 94fcf76e-1b18-53ae-a62f-86c68c9376cd with identifier: 3347506, dropping item
2026-04-01 06:50:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item e0e32956-d599-5102-ab78-5e0eb729fd45 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33497/3349613/93302849_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:50:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item e0e32956-d599-5102-ab78-5e0eb729fd45 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33497/3349613/93302873_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:50:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/rader-reifen-paket-10-satz-3361961/details> (referer: None)
2026-04-01 06:50:27 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: b83554bd-03cd-51ab-83ea-cb7b66ff1b01 with identifier: 3361961
2026-04-01 06:50:27 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b83554bd-03cd-51ab-83ea-cb7b66ff1b01 with scrape type 1
2026-04-01 06:50:27 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item b83554bd-03cd-51ab-83ea-cb7b66ff1b01
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33620%252F3361961%252F93454892_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:50:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33620/3361961/93454892_hd.jpg not downloaded yet
2026-04-01 06:50:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=47> (referer: None)
2026-04-01 06:50:27 [autobid.de] INFO: Page 47/145: found 20 items
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361964%27 HTTP/1.1" 200 None
2026-04-01 06:50:27 [autobid.de] INFO: Scrape type for 3361964: 1
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361962%27 HTTP/1.1" 200 None
2026-04-01 06:50:27 [autobid.de] INFO: Scrape type for 3361962: 1
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361963%27 HTTP/1.1" 200 None
2026-04-01 06:50:27 [autobid.de] INFO: Scrape type for 3361963: 1
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354018%27 HTTP/1.1" 200 None
2026-04-01 06:50:27 [autobid.de] INFO: Scrape type for 3354018: 1
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359719%27 HTTP/1.1" 200 None
2026-04-01 06:50:27 [autobid.de] INFO: Scrape type for 3359719: 1
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358925%27 HTTP/1.1" 200 None
2026-04-01 06:50:27 [autobid.de] INFO: Scrape type for 3358925: 1
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363209%27 HTTP/1.1" 200 None
2026-04-01 06:50:27 [autobid.de] INFO: Scrape type for 3363209: 1
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366412%27 HTTP/1.1" 200 None
2026-04-01 06:50:27 [autobid.de] INFO: Scrape type for 3366412: 1
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366485%27 HTTP/1.1" 200 None
2026-04-01 06:50:27 [autobid.de] INFO: Scrape type for 3366485: 1
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352525%27 HTTP/1.1" 200 None
2026-04-01 06:50:27 [autobid.de] INFO: Scrape type for 3352525: 1
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357677%27 HTTP/1.1" 200 None
2026-04-01 06:50:27 [autobid.de] INFO: Scrape type for 3357677: 1
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353488%27 HTTP/1.1" 200 None
2026-04-01 06:50:27 [autobid.de] INFO: Scrape type for 3353488: 1
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364213%27 HTTP/1.1" 200 None
2026-04-01 06:50:27 [autobid.de] INFO: Scrape type for 3364213: 1
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359843%27 HTTP/1.1" 200 None
2026-04-01 06:50:27 [autobid.de] INFO: Scrape type for 3359843: 1
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366102%27 HTTP/1.1" 200 None
2026-04-01 06:50:27 [autobid.de] INFO: Scrape type for 3366102: 1
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365143%27 HTTP/1.1" 200 None
2026-04-01 06:50:27 [autobid.de] INFO: Scrape type for 3365143: 1
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366106%27 HTTP/1.1" 200 None
2026-04-01 06:50:27 [autobid.de] INFO: Scrape type for 3366106: 1
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366434%27 HTTP/1.1" 200 None
2026-04-01 06:50:27 [autobid.de] INFO: Scrape type for 3366434: 1
2026-04-01 06:50:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362827%27 HTTP/1.1" 200 None
2026-04-01 06:50:28 [autobid.de] INFO: Scrape type for 3362827: 1
2026-04-01 06:50:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363615%27 HTTP/1.1" 200 None
2026-04-01 06:50:28 [autobid.de] INFO: Scrape type for 3363615: 1
2026-04-01 06:50:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 6f9f5d2d-b377-5cf3-9234-359ced998497 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33582/3358148/93389376_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:50:28 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/rader-reifen-paket-10-satz-3361965/details> (referer: None)
2026-04-01 06:50:28 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 55405799-b763-5e68-b93b-0cf722b020d0 with identifier: 3361965
2026-04-01 06:50:28 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 55405799-b763-5e68-b93b-0cf722b020d0 with scrape type 1
2026-04-01 06:50:28 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 55405799-b763-5e68-b93b-0cf722b020d0
2026-04-01 06:50:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33620%252F3361965%252F93454936_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:50:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33620/3361965/93454936_hd.jpg not downloaded yet
2026-04-01 06:50:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 19e8bd11-a744-5a9a-aae9-151cb9ff087b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33582/3358152/93386195_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:50:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/bmw-116i-m-sport-3363615/details> (referer: None)
2026-04-01 06:50:31 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 23bf71f5-d6bc-5a04-9c73-8d3f00152c45 with identifier: 3363615
2026-04-01 06:50:31 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 23bf71f5-d6bc-5a04-9c73-8d3f00152c45 with scrape type 1
2026-04-01 06:50:31 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 23bf71f5-d6bc-5a04-9c73-8d3f00152c45
2026-04-01 06:50:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33637%252F3363615%252F93525412_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:50:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33637/3363615/93525412_hd.jpg not downloaded yet
2026-04-01 06:50:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33637%252F3363615%252F93525405_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:50:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33637/3363615/93525405_hd.jpg not downloaded yet
2026-04-01 06:50:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33637%252F3363615%252F93525399_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:50:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33637/3363615/93525399_hd.jpg not downloaded yet
2026-04-01 06:50:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33637%252F3363615%252F93525402_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:50:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33637/3363615/93525402_hd.jpg not downloaded yet
2026-04-01 06:50:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33637%252F3363615%252F93525411_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:50:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33637/3363615/93525411_hd.jpg not downloaded yet
2026-04-01 06:50:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 19e8bd11-a744-5a9a-aae9-151cb9ff087b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33582/3358152/93386199_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:50:32 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/seat-tarraco-4drive-2-0-tdi-dsg-style-3362827/details> (referer: None)
2026-04-01 06:50:32 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=48> (referer: None)
2026-04-01 06:50:32 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 82198daf-caef-569c-aaa0-a2e4bf9e86d0 with identifier: 3362827
2026-04-01 06:50:32 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 82198daf-caef-569c-aaa0-a2e4bf9e86d0 with scrape type 1
2026-04-01 06:50:32 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 82198daf-caef-569c-aaa0-a2e4bf9e86d0
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33629%252F3362827%252F93628280_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:50:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33629/3362827/93628280_hd.jpg not downloaded yet
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33629%252F3362827%252F93628282_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:50:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33629/3362827/93628282_hd.jpg not downloaded yet
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33629%252F3362827%252F93628546_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:50:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33629/3362827/93628546_hd.jpg not downloaded yet
2026-04-01 06:50:32 [autobid.de] INFO: Page 48/145: found 20 items
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366484%27 HTTP/1.1" 200 None
2026-04-01 06:50:32 [autobid.de] INFO: Scrape type for 3366484: 1
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33629%252F3362827%252F93628544_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:50:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33629/3362827/93628544_hd.jpg not downloaded yet
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361029%27 HTTP/1.1" 200 None
2026-04-01 06:50:32 [autobid.de] INFO: Scrape type for 3361029: 1
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33629%252F3362827%252F93628284_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:50:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33629/3362827/93628284_hd.jpg not downloaded yet
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358937%27 HTTP/1.1" 200 None
2026-04-01 06:50:32 [autobid.de] INFO: Scrape type for 3358937: 1
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366399%27 HTTP/1.1" 200 None
2026-04-01 06:50:32 [autobid.de] INFO: Scrape type for 3366399: 1
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366425%27 HTTP/1.1" 200 None
2026-04-01 06:50:32 [autobid.de] INFO: Scrape type for 3366425: 1
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366487%27 HTTP/1.1" 200 None
2026-04-01 06:50:32 [autobid.de] INFO: Scrape type for 3366487: 1
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363013%27 HTTP/1.1" 200 None
2026-04-01 06:50:32 [autobid.de] INFO: Scrape type for 3363013: 1
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343417%27 HTTP/1.1" 200 None
2026-04-01 06:50:32 [autobid.de] INFO: Scrape type for 3343417: 1
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364214%27 HTTP/1.1" 200 None
2026-04-01 06:50:32 [autobid.de] INFO: Scrape type for 3364214: 1
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359960%27 HTTP/1.1" 200 None
2026-04-01 06:50:32 [autobid.de] INFO: Scrape type for 3359960: 1
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366103%27 HTTP/1.1" 200 None
2026-04-01 06:50:32 [autobid.de] INFO: Scrape type for 3366103: 1
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365142%27 HTTP/1.1" 200 None
2026-04-01 06:50:32 [autobid.de] INFO: Scrape type for 3365142: 1
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366107%27 HTTP/1.1" 200 None
2026-04-01 06:50:32 [autobid.de] INFO: Scrape type for 3366107: 1
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366435%27 HTTP/1.1" 200 None
2026-04-01 06:50:32 [autobid.de] INFO: Scrape type for 3366435: 1
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366440%27 HTTP/1.1" 200 None
2026-04-01 06:50:32 [autobid.de] INFO: Scrape type for 3366440: 1
2026-04-01 06:50:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366431%27 HTTP/1.1" 200 None
2026-04-01 06:50:33 [autobid.de] INFO: Scrape type for 3366431: 1
2026-04-01 06:50:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345589%27 HTTP/1.1" 200 None
2026-04-01 06:50:33 [autobid.de] INFO: Scrape type for 3345589: 1
2026-04-01 06:50:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273350066%27 HTTP/1.1" 200 None
2026-04-01 06:50:33 [autobid.de] INFO: Scrape type for 3350066: 1
2026-04-01 06:50:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362662%27 HTTP/1.1" 200 None
2026-04-01 06:50:33 [autobid.de] INFO: Scrape type for 3362662: 1
2026-04-01 06:50:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:50:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359721%27 HTTP/1.1" 200 None
2026-04-01 06:50:33 [autobid.de] INFO: Scrape type for 3359721: 1
2026-04-01 06:50:42 [scrapy.extensions.logstats] INFO: Crawled 189 pages (at 10 pages/min), scraped 8 items (at 0 items/min)
2026-04-01 06:50:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 240MiB
2026-04-01 06:51:42 [scrapy.extensions.logstats] INFO: Crawled 189 pages (at 0 pages/min), scraped 8 items (at 0 items/min)
2026-04-01 06:51:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 240MiB
2026-04-01 06:52:42 [scrapy.extensions.logstats] INFO: Crawled 189 pages (at 0 pages/min), scraped 8 items (at 0 items/min)
2026-04-01 06:52:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 240MiB
2026-04-01 06:53:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 502bcdde-4897-5a33-b3f3-b41bb977e2de due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33582/3358137/93453227_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:53:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 90f4c135-3398-57cf-be61-7a8401217c1b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33574/3357392/93425243_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:53:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-tt-coupe-2-0-tdi-ultra-s-line-3359721/details> (referer: None)
2026-04-01 06:53:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 2394bf60-753c-525e-83e3-2562f363c7b9 with identifier: 3359721
2026-04-01 06:53:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 2394bf60-753c-525e-83e3-2562f363c7b9 with scrape type 1
2026-04-01 06:53:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 2394bf60-753c-525e-83e3-2562f363c7b9
2026-04-01 06:53:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33598%252F3359721%252F93408897_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33598/3359721/93408897_hd.jpg not downloaded yet
2026-04-01 06:53:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33598%252F3359721%252F93408899_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33598/3359721/93408899_hd.jpg not downloaded yet
2026-04-01 06:53:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33598%252F3359721%252F93408624_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33598/3359721/93408624_hd.jpg not downloaded yet
2026-04-01 06:53:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33598%252F3359721%252F93408625_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33598/3359721/93408625_hd.jpg not downloaded yet
2026-04-01 06:53:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33598%252F3359721%252F93408626_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33598/3359721/93408626_hd.jpg not downloaded yet
2026-04-01 06:53:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=49> (referer: None)
2026-04-01 06:53:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/bmw-m8-competition-xdrive-cabriolet-sport-automatic-3362662/details> (referer: None)
2026-04-01 06:53:23 [autobid.de] INFO: Page 49/145: found 20 items
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366488%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [autobid.de] INFO: Scrape type for 3366488: 1
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273341635%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [autobid.de] INFO: Scrape type for 3341635: 1
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363012%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [autobid.de] INFO: Scrape type for 3363012: 1
2026-04-01 06:53:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: d47eab94-7a9f-5ec7-924d-ea51d49d71ae with identifier: 3362662
2026-04-01 06:53:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item d47eab94-7a9f-5ec7-924d-ea51d49d71ae with scrape type 1
2026-04-01 06:53:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item d47eab94-7a9f-5ec7-924d-ea51d49d71ae
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362662%252F93480632_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362662/93480632_hd.jpg not downloaded yet
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361708%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [autobid.de] INFO: Scrape type for 3361708: 1
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362662%252F93480649_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362662/93480649_hd.jpg not downloaded yet
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359988%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [autobid.de] INFO: Scrape type for 3359988: 1
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362662%252F93480626_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362662/93480626_hd.jpg not downloaded yet
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363394%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [autobid.de] INFO: Scrape type for 3363394: 1
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362662%252F93480637_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362662/93480637_hd.jpg not downloaded yet
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365144%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [autobid.de] INFO: Scrape type for 3365144: 1
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362662%252F93480652_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362662/93480652_hd.jpg not downloaded yet
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366108%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [autobid.de] INFO: Scrape type for 3366108: 1
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366436%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [autobid.de] INFO: Scrape type for 3366436: 1
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358258%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [autobid.de] INFO: Scrape type for 3358258: 1
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345590%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [autobid.de] INFO: Scrape type for 3345590: 1
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352115%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [autobid.de] INFO: Scrape type for 3352115: 1
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366452%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [autobid.de] INFO: Scrape type for 3366452: 1
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361027%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [autobid.de] INFO: Scrape type for 3361027: 1
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358921%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [autobid.de] INFO: Scrape type for 3358921: 1
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366408%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [autobid.de] INFO: Scrape type for 3366408: 1
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366428%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [autobid.de] INFO: Scrape type for 3366428: 1
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366489%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [autobid.de] INFO: Scrape type for 3366489: 1
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363014%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [autobid.de] INFO: Scrape type for 3363014: 1
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366442%27 HTTP/1.1" 200 None
2026-04-01 06:53:23 [autobid.de] INFO: Scrape type for 3366442: 1
2026-04-01 06:53:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 90f4c135-3398-57cf-be61-7a8401217c1b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33574/3357392/93425242_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:53:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 90f4c135-3398-57cf-be61-7a8401217c1b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33574/3357392/93425241_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:53:25 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-sq8-sportback-e-tron-quattro-cont-226-kw-dual-elektromotor-3366442/details> (referer: None)
2026-04-01 06:53:25 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: fc4054a6-02e3-542b-98a4-9dc33969f2e6 with identifier: 3366442
2026-04-01 06:53:25 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item fc4054a6-02e3-542b-98a4-9dc33969f2e6 with scrape type 1
2026-04-01 06:53:25 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item fc4054a6-02e3-542b-98a4-9dc33969f2e6
2026-04-01 06:53:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366442%252F93631633_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366442/93631633_hd.jpg not downloaded yet
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366442%252F93631632_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366442/93631632_hd.jpg not downloaded yet
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366442%252F93631629_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366442/93631629_hd.jpg not downloaded yet
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366442%252F93631624_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366442/93631624_hd.jpg not downloaded yet
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366442%252F93631622_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366442/93631622_hd.jpg not downloaded yet
2026-04-01 06:53:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=50> (referer: None)
2026-04-01 06:53:26 [autobid.de] INFO: Page 50/145: found 20 items
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273349505%27 HTTP/1.1" 200 None
2026-04-01 06:53:26 [autobid.de] INFO: Scrape type for 3349505: 1
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364215%27 HTTP/1.1" 200 None
2026-04-01 06:53:26 [autobid.de] INFO: Scrape type for 3364215: 1
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362674%27 HTTP/1.1" 200 None
2026-04-01 06:53:26 [autobid.de] INFO: Scrape type for 3362674: 1
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366104%27 HTTP/1.1" 200 None
2026-04-01 06:53:26 [autobid.de] INFO: Scrape type for 3366104: 1
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365145%27 HTTP/1.1" 200 None
2026-04-01 06:53:26 [autobid.de] INFO: Scrape type for 3365145: 1
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366109%27 HTTP/1.1" 200 None
2026-04-01 06:53:26 [autobid.de] INFO: Scrape type for 3366109: 1
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366437%27 HTTP/1.1" 200 None
2026-04-01 06:53:26 [autobid.de] INFO: Scrape type for 3366437: 1
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365152%27 HTTP/1.1" 200 None
2026-04-01 06:53:26 [autobid.de] INFO: Scrape type for 3365152: 1
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353490%27 HTTP/1.1" 200 None
2026-04-01 06:53:26 [autobid.de] INFO: Scrape type for 3353490: 1
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354019%27 HTTP/1.1" 200 None
2026-04-01 06:53:26 [autobid.de] INFO: Scrape type for 3354019: 1
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359720%27 HTTP/1.1" 200 None
2026-04-01 06:53:26 [autobid.de] INFO: Scrape type for 3359720: 1
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358939%27 HTTP/1.1" 200 None
2026-04-01 06:53:26 [autobid.de] INFO: Scrape type for 3358939: 1
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366414%27 HTTP/1.1" 200 None
2026-04-01 06:53:26 [autobid.de] INFO: Scrape type for 3366414: 1
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366429%27 HTTP/1.1" 200 None
2026-04-01 06:53:26 [autobid.de] INFO: Scrape type for 3366429: 1
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366490%27 HTTP/1.1" 200 None
2026-04-01 06:53:26 [autobid.de] INFO: Scrape type for 3366490: 1
2026-04-01 06:53:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352522%27 HTTP/1.1" 200 None
2026-04-01 06:53:27 [autobid.de] INFO: Scrape type for 3352522: 1
2026-04-01 06:53:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357678%27 HTTP/1.1" 200 None
2026-04-01 06:53:27 [autobid.de] INFO: Scrape type for 3357678: 1
2026-04-01 06:53:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355033%27 HTTP/1.1" 200 None
2026-04-01 06:53:27 [autobid.de] INFO: Scrape type for 3355033: 1
2026-04-01 06:53:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361662%27 HTTP/1.1" 200 None
2026-04-01 06:53:27 [autobid.de] INFO: Scrape type for 3361662: 1
2026-04-01 06:53:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366105%27 HTTP/1.1" 200 None
2026-04-01 06:53:27 [autobid.de] INFO: Scrape type for 3366105: 1
2026-04-01 06:53:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 31416781-87c9-51f7-b738-094cdc420be1 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362656/93530326_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:53:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-q5-advanced-50-tfsi-e-quattro-s-tronic-7-gang-doppelkupplungsgetriebe-s-tronic-3363014/details> (referer: None)
2026-04-01 06:53:27 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 6dd5cfad-2f25-5699-9d5c-bb1a1dc5b49e with identifier: 3363014
2026-04-01 06:53:27 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6dd5cfad-2f25-5699-9d5c-bb1a1dc5b49e with scrape type 1
2026-04-01 06:53:27 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 6dd5cfad-2f25-5699-9d5c-bb1a1dc5b49e
2026-04-01 06:53:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33631%252F3363014%252F93488617_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33631/3363014/93488617_hd.jpg not downloaded yet
2026-04-01 06:53:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33631%252F3363014%252F93488605_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33631/3363014/93488605_hd.jpg not downloaded yet
2026-04-01 06:53:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33631%252F3363014%252F93488625_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33631/3363014/93488625_hd.jpg not downloaded yet
2026-04-01 06:53:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33631%252F3363014%252F93488623_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33631/3363014/93488623_hd.jpg not downloaded yet
2026-04-01 06:53:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33631%252F3363014%252F93488611_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33631/3363014/93488611_hd.jpg not downloaded yet
2026-04-01 06:53:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 7d5021fd-a243-5632-b20a-e47e311152f8 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33583/3358276/93488078_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:53:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-passat-variant-2-0-tdi-dsg-highline-3366105/details> (referer: None)
2026-04-01 06:53:30 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 3c126531-3c4c-59f5-9b66-0b899016556a with identifier: 3366105
2026-04-01 06:53:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 3c126531-3c4c-59f5-9b66-0b899016556a with scrape type 1
2026-04-01 06:53:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 3c126531-3c4c-59f5-9b66-0b899016556a
2026-04-01 06:53:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33662%252F3366105%252F93619536_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33662/3366105/93619536_hd.jpg not downloaded yet
2026-04-01 06:53:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33662%252F3366105%252F93619544_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33662/3366105/93619544_hd.jpg not downloaded yet
2026-04-01 06:53:30 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 94fcf76e-1b18-53ae-a62f-86c68c9376cd with identifier: 3347495
2026-04-01 06:53:30 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 06:53:30 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=Unknown HTTP/1.1" 200 None
2026-04-01 06:53:30 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 06:53:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 06:53:30 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 06:53:30 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=other HTTP/1.1" 200 None
2026-04-01 06:53:30 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 06:53:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 06:53:30 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 94fcf76e-1b18-53ae-a62f-86c68c9376cd with identifier: 3347495
2026-04-01 06:53:30 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Dropping item: 94fcf76e-1b18-53ae-a62f-86c68c9376cd with missing fields: registration_date, year
2026-04-01 06:53:30 [scrapy.core.scraper] INFO: Dropped: Spider: autobid.de, Dropping item: 94fcf76e-1b18-53ae-a62f-86c68c9376cd with missing fields: registration_date, year
{'auction_title': 'Auction 81218',
'body_type': 'Other',
'category': 'Other',
'color': 'Unknown',
'currency': 'EUR',
'engine': '0.1L',
'engine_horse_power': 1,
'expiration_date': '2026-04-02T07:00:00Z',
'fuel_type': 'other',
'id': '94fcf76e-1b18-53ae-a62f-86c68c9376cd',
'is_damaged': False,
'is_operable': True,
'is_vat_included': True,
'link_to_web_offer': 'https://autobid.de/en/item/rader-reifen-paket-10-satz-3347495',
'listing': 'auction',
'location': 'Hamburg',
'make': 'Rader / Reifen',
'model': 'Räder / Reifen Paket',
'odometer_reading_unit': 'km',
'origin_country_code': 'DE',
'original_photo_count': 1,
'photo_urls': ['https://cdn.autobid.de/data/cars/33475/3347495/93143265_hd.jpg'],
'photos': [],
'price': 400,
'price_includes_vat': True,
'seller_name': 'Audi Hamburg GmbH',
'title': 'Räder / Reifen Paket 10 Satz',
'transmission': 'manual',
'vin': None}
2026-04-01 06:53:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273347495%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [autobid.de] INFO: Saving data for 3347495: {'created_time': 1775026411.002393, 'last_price_update_time': 1775026411.002403, 'auction_closing_time': 1775113200.0}
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3347495') HTTP/1.1" 204 0
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33662%252F3366105%252F93619559_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33662/3366105/93619559_hd.jpg not downloaded yet
2026-04-01 06:53:31 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 94fcf76e-1b18-53ae-a62f-86c68c9376cd due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33475/3347495/93143265_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33662%252F3366105%252F93619551_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33662/3366105/93619551_hd.jpg not downloaded yet
2026-04-01 06:53:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=51> (referer: None)
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33662%252F3366105%252F93619547_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33662/3366105/93619547_hd.jpg not downloaded yet
2026-04-01 06:53:31 [autobid.de] INFO: Page 51/145: found 20 items
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366110%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [autobid.de] INFO: Scrape type for 3366110: 1
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366438%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [autobid.de] INFO: Scrape type for 3366438: 1
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366441%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [autobid.de] INFO: Scrape type for 3366441: 1
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352573%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [autobid.de] INFO: Scrape type for 3352573: 1
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343953%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [autobid.de] INFO: Scrape type for 3343953: 1
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366439%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [autobid.de] INFO: Scrape type for 3366439: 1
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361028%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [autobid.de] INFO: Scrape type for 3361028: 1
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358917%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [autobid.de] INFO: Scrape type for 3358917: 1
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366422%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [autobid.de] INFO: Scrape type for 3366422: 1
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366430%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [autobid.de] INFO: Scrape type for 3366430: 1
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366491%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [autobid.de] INFO: Scrape type for 3366491: 1
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366486%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [autobid.de] INFO: Scrape type for 3366486: 1
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355034%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [autobid.de] INFO: Scrape type for 3355034: 1
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364216%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [autobid.de] INFO: Scrape type for 3364216: 1
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362679%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [autobid.de] INFO: Scrape type for 3362679: 1
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363396%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [autobid.de] INFO: Scrape type for 3363396: 1
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365147%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [autobid.de] INFO: Scrape type for 3365147: 1
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366111%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [autobid.de] INFO: Scrape type for 3366111: 1
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366400%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [autobid.de] INFO: Scrape type for 3366400: 1
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273159974%27 HTTP/1.1" 200 None
2026-04-01 06:53:31 [autobid.de] INFO: Scrape type for 3159974: 1
2026-04-01 06:53:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 7d5021fd-a243-5632-b20a-e47e311152f8 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33583/3358276/93488077_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:53:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=52> (referer: None)
2026-04-01 06:53:34 [autobid.de] INFO: Page 52/145: found 20 items
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366406%27 HTTP/1.1" 200 None
2026-04-01 06:53:34 [autobid.de] INFO: Scrape type for 3366406: 1
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355035%27 HTTP/1.1" 200 None
2026-04-01 06:53:34 [autobid.de] INFO: Scrape type for 3355035: 1
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343960%27 HTTP/1.1" 200 None
2026-04-01 06:53:34 [autobid.de] INFO: Scrape type for 3343960: 1
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359987%27 HTTP/1.1" 200 None
2026-04-01 06:53:34 [autobid.de] INFO: Scrape type for 3359987: 1
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366112%27 HTTP/1.1" 200 None
2026-04-01 06:53:34 [autobid.de] INFO: Scrape type for 3366112: 1
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358928%27 HTTP/1.1" 200 None
2026-04-01 06:53:34 [autobid.de] INFO: Scrape type for 3358928: 1
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366188%27 HTTP/1.1" 200 None
2026-04-01 06:53:34 [autobid.de] INFO: Scrape type for 3366188: 1
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366402%27 HTTP/1.1" 200 None
2026-04-01 06:53:34 [autobid.de] INFO: Scrape type for 3366402: 1
2026-04-01 06:53:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/man-tgl-12-250-3159974/details> (referer: None)
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366463%27 HTTP/1.1" 200 None
2026-04-01 06:53:34 [autobid.de] INFO: Scrape type for 3366463: 1
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352521%27 HTTP/1.1" 200 None
2026-04-01 06:53:34 [autobid.de] INFO: Scrape type for 3352521: 1
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357355%27 HTTP/1.1" 200 None
2026-04-01 06:53:34 [autobid.de] INFO: Scrape type for 3357355: 1
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358363%27 HTTP/1.1" 200 None
2026-04-01 06:53:34 [autobid.de] INFO: Scrape type for 3358363: 1
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364217%27 HTTP/1.1" 200 None
2026-04-01 06:53:34 [autobid.de] INFO: Scrape type for 3364217: 1
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359986%27 HTTP/1.1" 200 None
2026-04-01 06:53:34 [autobid.de] INFO: Scrape type for 3359986: 1
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366113%27 HTTP/1.1" 200 None
2026-04-01 06:53:34 [autobid.de] INFO: Scrape type for 3366113: 1
2026-04-01 06:53:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 0c0aa48a-bbfd-53a4-b265-6bd7d1f2e34e with identifier: 3159974
2026-04-01 06:53:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 0c0aa48a-bbfd-53a4-b265-6bd7d1f2e34e with scrape type 1
2026-04-01 06:53:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 0c0aa48a-bbfd-53a4-b265-6bd7d1f2e34e
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F31600%252F3159974%252F86506007_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/31600/3159974/86506007_hd.jpg not downloaded yet
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365148%27 HTTP/1.1" 200 None
2026-04-01 06:53:34 [autobid.de] INFO: Scrape type for 3365148: 1
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F31600%252F3159974%252F86506008_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/31600/3159974/86506008_hd.jpg not downloaded yet
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366398%27 HTTP/1.1" 200 None
2026-04-01 06:53:34 [autobid.de] INFO: Scrape type for 3366398: 1
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F31600%252F3159974%252F86506009_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/31600/3159974/86506009_hd.jpg not downloaded yet
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366419%27 HTTP/1.1" 200 None
2026-04-01 06:53:34 [autobid.de] INFO: Scrape type for 3366419: 1
2026-04-01 06:53:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F31600%252F3159974%252F86506010_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/31600/3159974/86506010_hd.jpg not downloaded yet
2026-04-01 06:53:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366460%27 HTTP/1.1" 200 None
2026-04-01 06:53:35 [autobid.de] INFO: Scrape type for 3366460: 1
2026-04-01 06:53:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F31600%252F3159974%252F86506011_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/31600/3159974/86506011_hd.jpg not downloaded yet
2026-04-01 06:53:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358364%27 HTTP/1.1" 200 None
2026-04-01 06:53:35 [autobid.de] INFO: Scrape type for 3358364: 1
2026-04-01 06:53:36 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/peugeot-408-gt-1-2-e-thp-automatik-3366402/details> (referer: None)
2026-04-01 06:53:36 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 06dabfeb-d0e2-5784-8c3f-3084f111f973 with identifier: 3366402
2026-04-01 06:53:36 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 06dabfeb-d0e2-5784-8c3f-3084f111f973 with scrape type 1
2026-04-01 06:53:36 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 06dabfeb-d0e2-5784-8c3f-3084f111f973
2026-04-01 06:53:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366402%252F93630371_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:36 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366402/93630371_hd.jpg not downloaded yet
2026-04-01 06:53:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366402%252F93630377_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:36 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366402/93630377_hd.jpg not downloaded yet
2026-04-01 06:53:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366402%252F93630366_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:36 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366402/93630366_hd.jpg not downloaded yet
2026-04-01 06:53:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366402%252F93630385_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:36 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366402/93630385_hd.jpg not downloaded yet
2026-04-01 06:53:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366402%252F93630370_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:53:36 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366402/93630370_hd.jpg not downloaded yet
2026-04-01 06:53:38 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=53> (referer: None)
2026-04-01 06:53:39 [autobid.de] INFO: Page 53/145: found 20 items
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343970%27 HTTP/1.1" 200 None
2026-04-01 06:53:39 [autobid.de] INFO: Scrape type for 3343970: 1
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366448%27 HTTP/1.1" 200 None
2026-04-01 06:53:39 [autobid.de] INFO: Scrape type for 3366448: 1
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363398%27 HTTP/1.1" 200 None
2026-04-01 06:53:39 [autobid.de] INFO: Scrape type for 3363398: 1
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358934%27 HTTP/1.1" 200 None
2026-04-01 06:53:39 [autobid.de] INFO: Scrape type for 3358934: 1
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366433%27 HTTP/1.1" 200 None
2026-04-01 06:53:39 [autobid.de] INFO: Scrape type for 3366433: 1
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366403%27 HTTP/1.1" 200 None
2026-04-01 06:53:39 [autobid.de] INFO: Scrape type for 3366403: 1
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366467%27 HTTP/1.1" 200 None
2026-04-01 06:53:39 [autobid.de] INFO: Scrape type for 3366467: 1
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366479%27 HTTP/1.1" 200 None
2026-04-01 06:53:39 [autobid.de] INFO: Scrape type for 3366479: 1
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366446%27 HTTP/1.1" 200 None
2026-04-01 06:53:39 [autobid.de] INFO: Scrape type for 3366446: 1
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345587%27 HTTP/1.1" 200 None
2026-04-01 06:53:39 [autobid.de] INFO: Scrape type for 3345587: 1
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361714%27 HTTP/1.1" 200 None
2026-04-01 06:53:39 [autobid.de] INFO: Scrape type for 3361714: 1
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363210%27 HTTP/1.1" 200 None
2026-04-01 06:53:39 [autobid.de] INFO: Scrape type for 3363210: 1
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366114%27 HTTP/1.1" 200 None
2026-04-01 06:53:39 [autobid.de] INFO: Scrape type for 3366114: 1
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366404%27 HTTP/1.1" 200 None
2026-04-01 06:53:39 [autobid.de] INFO: Scrape type for 3366404: 1
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366420%27 HTTP/1.1" 200 None
2026-04-01 06:53:39 [autobid.de] INFO: Scrape type for 3366420: 1
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366407%27 HTTP/1.1" 200 None
2026-04-01 06:53:39 [autobid.de] INFO: Scrape type for 3366407: 1
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366535%27 HTTP/1.1" 200 None
2026-04-01 06:53:39 [autobid.de] INFO: Scrape type for 3366535: 1
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366464%27 HTTP/1.1" 200 None
2026-04-01 06:53:39 [autobid.de] INFO: Scrape type for 3366464: 1
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343961%27 HTTP/1.1" 200 None
2026-04-01 06:53:39 [autobid.de] INFO: Scrape type for 3343961: 1
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:53:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361663%27 HTTP/1.1" 200 None
2026-04-01 06:53:39 [autobid.de] INFO: Scrape type for 3361663: 1
2026-04-01 06:53:42 [scrapy.extensions.logstats] INFO: Crawled 201 pages (at 12 pages/min), scraped 8 items (at 0 items/min)
2026-04-01 06:53:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 241MiB
2026-04-01 06:54:42 [scrapy.extensions.logstats] INFO: Crawled 201 pages (at 0 pages/min), scraped 8 items (at 0 items/min)
2026-04-01 06:54:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 241MiB
2026-04-01 06:55:42 [scrapy.extensions.logstats] INFO: Crawled 201 pages (at 0 pages/min), scraped 8 items (at 0 items/min)
2026-04-01 06:55:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 241MiB
2026-04-01 06:56:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 14193e74-947e-5162-9770-5016e30f258b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33619/3361826/93455543_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:56:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 14193e74-947e-5162-9770-5016e30f258b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33619/3361826/93455542_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:56:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-a4-avant-40-2-0-tfsi-s-tronic-3361663/details> (referer: None)
2026-04-01 06:56:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: ccb5e478-91ff-58f3-b091-aa0fc6e3101c with identifier: 3361663
2026-04-01 06:56:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ccb5e478-91ff-58f3-b091-aa0fc6e3101c with scrape type 1
2026-04-01 06:56:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item ccb5e478-91ff-58f3-b091-aa0fc6e3101c
2026-04-01 06:56:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33617%252F3361663%252F93475179_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:56:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33617/3361663/93475179_hd.jpg not downloaded yet
2026-04-01 06:56:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33617%252F3361663%252F93475180_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:56:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33617/3361663/93475180_hd.jpg not downloaded yet
2026-04-01 06:56:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33617%252F3361663%252F93475176_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:56:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33617/3361663/93475176_hd.jpg not downloaded yet
2026-04-01 06:56:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33617%252F3361663%252F93475175_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:56:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33617/3361663/93475175_hd.jpg not downloaded yet
2026-04-01 06:56:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33617%252F3361663%252F93475173_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:56:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33617/3361663/93475173_hd.jpg not downloaded yet
2026-04-01 06:56:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=54> (referer: None)
2026-04-01 06:56:23 [autobid.de] INFO: Page 54/145: found 20 items
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366413%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [autobid.de] INFO: Scrape type for 3366413: 1
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358932%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [autobid.de] INFO: Scrape type for 3358932: 1
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366462%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [autobid.de] INFO: Scrape type for 3366462: 1
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366409%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [autobid.de] INFO: Scrape type for 3366409: 1
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366470%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [autobid.de] INFO: Scrape type for 3366470: 1
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357030%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [autobid.de] INFO: Scrape type for 3357030: 1
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352590%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [autobid.de] INFO: Scrape type for 3352590: 1
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353321%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [autobid.de] INFO: Scrape type for 3353321: 1
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359991%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [autobid.de] INFO: Scrape type for 3359991: 1
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363400%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [autobid.de] INFO: Scrape type for 3363400: 1
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366410%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [autobid.de] INFO: Scrape type for 3366410: 1
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366449%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [autobid.de] INFO: Scrape type for 3366449: 1
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366483%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [autobid.de] INFO: Scrape type for 3366483: 1
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366536%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [autobid.de] INFO: Scrape type for 3366536: 1
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361710%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [autobid.de] INFO: Scrape type for 3361710: 1
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366857%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [autobid.de] INFO: Scrape type for 3366857: 1
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343951%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [autobid.de] INFO: Scrape type for 3343951: 1
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366447%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [autobid.de] INFO: Scrape type for 3366447: 1
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366416%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [autobid.de] INFO: Scrape type for 3366416: 1
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358924%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [autobid.de] INFO: Scrape type for 3358924: 1
2026-04-01 06:56:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ford-transit-doka-pritsche-350-l2-2-0-tdci-trend-3343961/details> (referer: None)
2026-04-01 06:56:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 3a019736-3659-5080-b7ff-4c4fc0eee3db with identifier: 3343961
2026-04-01 06:56:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 3a019736-3659-5080-b7ff-4c4fc0eee3db with scrape type 1
2026-04-01 06:56:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 3a019736-3659-5080-b7ff-4c4fc0eee3db
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343961%252F92830906_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343961/92830906_hd.jpg not downloaded yet
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343961%252F92830903_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343961/92830903_hd.jpg not downloaded yet
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343961%252F92830917_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343961/92830917_hd.jpg not downloaded yet
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343961%252F92830919_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343961/92830919_hd.jpg not downloaded yet
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343961%252F92830905_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:56:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343961/92830905_hd.jpg not downloaded yet
2026-04-01 06:56:24 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 14193e74-947e-5162-9770-5016e30f258b with identifier: 3361826
2026-04-01 06:56:24 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 14193e74-947e-5162-9770-5016e30f258b with identifier: 3361826
2026-04-01 06:56:24 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: 14193e74-947e-5162-9770-5016e30f258b with identifier: 3361826 to the API
2026-04-01 06:56:24 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 06:56:24 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 06:56:24 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33619/3361826/93455541_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 81469',
'body_type': 'All-terrain vehicle',
'category': 'All-terrain vehicle',
'color': 'grey Metallic',
'currency': 'EUR',
'emission_standard': 'EURO 6',
'engine': '3.0L',
'engine_horse_power': 286,
'expiration_date': '2026-04-02T07:00:00Z',
'fuel_type': 'Diesel',
'id': '14193e74-947e-5162-9770-5016e30f258b',
'is_damaged': True,
'is_operable': True,
'is_vat_included': True,
'link_to_web_offer': 'https://autobid.de/en/item/bmw-x5-30-d-xdrive-m-sport-connected-professional-uvp-127-490-00-3361826',
'listing': 'auction',
'location': 'Landsberg OT Peißen',
'make': 'BMW',
'model': 'X5',
'odometer': 81100,
'odometer_reading_unit': 'km',
'origin_country_code': 'DE',
'original_photo_count': 3,
'photo_urls': ['https://cdn.autobid.de/data/cars/33619/3361826/93455541_hd.jpg',
'https://cdn.autobid.de/data/cars/33619/3361826/93455542_hd.jpg',
'https://cdn.autobid.de/data/cars/33619/3361826/93455543_hd.jpg'],
'photos': [],
'price': 52800,
'price_includes_vat': True,
'registration_date': '2024-09-01',
'seats_number': 5,
'seller_name': None,
'title': 'BMW X5 30 d xDrive M Sport Connected Professional UVP: 127.490,00 €',
'transmission': 'automatic',
'vin': None,
'year': 2024}
2026-04-01 06:56:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361826%27 HTTP/1.1" 200 None
2026-04-01 06:56:24 [autobid.de] INFO: Saving data for 3361826: {'created_time': 1775026584.986419, 'last_price_update_time': 1775026584.986431, 'auction_closing_time': 1775113200.0}
2026-04-01 06:56:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3361826') HTTP/1.1" 204 0
2026-04-01 06:56:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 14193e74-947e-5162-9770-5016e30f258b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33619/3361826/93455541_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:56:25 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ford-fiesta-1-0-ecoboost-automatik-titanium-3358924/details> (referer: None)
2026-04-01 06:56:25 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 7325dbfd-c6dd-51bb-937f-40f46218babe with identifier: 3358924
2026-04-01 06:56:25 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 7325dbfd-c6dd-51bb-937f-40f46218babe with scrape type 1
2026-04-01 06:56:25 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 7325dbfd-c6dd-51bb-937f-40f46218babe
2026-04-01 06:56:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33590%252F3358924%252F93371006_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:56:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33590/3358924/93371006_hd.jpg not downloaded yet
2026-04-01 06:56:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33590%252F3358924%252F93371005_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:56:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33590/3358924/93371005_hd.jpg not downloaded yet
2026-04-01 06:56:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33590%252F3358924%252F93370738_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:56:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33590/3358924/93370738_hd.jpg not downloaded yet
2026-04-01 06:56:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33590%252F3358924%252F93370739_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:56:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33590/3358924/93370739_hd.jpg not downloaded yet
2026-04-01 06:56:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33590%252F3358924%252F93370740_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:56:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33590/3358924/93370740_hd.jpg not downloaded yet
2026-04-01 06:56:25 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: b83554bd-03cd-51ab-83ea-cb7b66ff1b01 with identifier: 3361961
2026-04-01 06:56:25 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: b83554bd-03cd-51ab-83ea-cb7b66ff1b01 with identifier: 3361961
2026-04-01 06:56:25 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Dropping item: b83554bd-03cd-51ab-83ea-cb7b66ff1b01 with missing fields: registration_date, year
2026-04-01 06:56:25 [scrapy.core.scraper] INFO: Dropped: Spider: autobid.de, Dropping item: b83554bd-03cd-51ab-83ea-cb7b66ff1b01 with missing fields: registration_date, year
{'auction_title': 'Auction 81218',
'body_type': 'Packages',
'category': 'Packages',
'color': 'Unknown',
'currency': 'EUR',
'engine': '0.1L',
'engine_horse_power': 1,
'expiration_date': '2026-04-02T07:00:00Z',
'fuel_type': 'other',
'id': 'b83554bd-03cd-51ab-83ea-cb7b66ff1b01',
'is_damaged': False,
'is_operable': True,
'is_vat_included': True,
'link_to_web_offer': 'https://autobid.de/en/item/rader-reifen-paket-10-satz-3361961',
'listing': 'auction',
'location': 'Hamburg',
'make': 'Rader / Reifen',
'model': 'Räder / Reifen Paket',
'odometer_reading_unit': 'km',
'origin_country_code': 'DE',
'original_photo_count': 1,
'photo_urls': ['https://cdn.autobid.de/data/cars/33620/3361961/93454892_hd.jpg'],
'photos': [],
'price': 1500,
'price_includes_vat': True,
'seller_name': 'Subsidiary/authorised dealer of the brand(s): Audi',
'title': 'Räder / Reifen Paket 10 Satz',
'transmission': 'manual',
'vin': None}
2026-04-01 06:56:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361961%27 HTTP/1.1" 200 None
2026-04-01 06:56:25 [autobid.de] INFO: Saving data for 3361961: {'created_time': 1775026585.812873, 'last_price_update_time': 1775026585.812882, 'auction_closing_time': 1775113200.0}
2026-04-01 06:56:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3361961') HTTP/1.1" 204 0
2026-04-01 06:56:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item b83554bd-03cd-51ab-83ea-cb7b66ff1b01 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33620/3361961/93454892_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:56:27 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 55405799-b763-5e68-b93b-0cf722b020d0 with identifier: 3361965
2026-04-01 06:56:27 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 55405799-b763-5e68-b93b-0cf722b020d0 with identifier: 3361965
2026-04-01 06:56:27 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Dropping item: 55405799-b763-5e68-b93b-0cf722b020d0 with missing fields: registration_date, year
2026-04-01 06:56:27 [scrapy.core.scraper] INFO: Dropped: Spider: autobid.de, Dropping item: 55405799-b763-5e68-b93b-0cf722b020d0 with missing fields: registration_date, year
{'auction_title': 'Auction 81218',
'body_type': 'Other',
'category': 'Other',
'color': 'Unknown',
'currency': 'EUR',
'engine': '0.1L',
'engine_horse_power': 1,
'expiration_date': '2026-04-02T07:00:00Z',
'fuel_type': 'other',
'id': '55405799-b763-5e68-b93b-0cf722b020d0',
'is_damaged': False,
'is_operable': True,
'is_vat_included': True,
'link_to_web_offer': 'https://autobid.de/en/item/rader-reifen-paket-10-satz-3361965',
'listing': 'auction',
'location': 'Hamburg',
'make': 'Rader / Reifen',
'model': 'Räder / Reifen Paket',
'odometer_reading_unit': 'km',
'origin_country_code': 'DE',
'original_photo_count': 1,
'photo_urls': ['https://cdn.autobid.de/data/cars/33620/3361965/93454936_hd.jpg'],
'photos': [],
'price': 2500,
'price_includes_vat': True,
'seller_name': 'Audi Hamburg GmbH',
'title': 'Räder / Reifen Paket 10 Satz',
'transmission': 'manual',
'vin': None}
2026-04-01 06:56:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361965%27 HTTP/1.1" 200 None
2026-04-01 06:56:27 [autobid.de] INFO: Saving data for 3361965: {'created_time': 1775026587.314365, 'last_price_update_time': 1775026587.314381, 'auction_closing_time': 1775113200.0}
2026-04-01 06:56:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:56:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3361965') HTTP/1.1" 204 0
2026-04-01 06:56:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 55405799-b763-5e68-b93b-0cf722b020d0 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33620/3361965/93454936_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:56:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 23bf71f5-d6bc-5a04-9c73-8d3f00152c45 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33637/3363615/93525411_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:56:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 23bf71f5-d6bc-5a04-9c73-8d3f00152c45 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33637/3363615/93525402_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:56:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 23bf71f5-d6bc-5a04-9c73-8d3f00152c45 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33637/3363615/93525399_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:56:42 [scrapy.extensions.logstats] INFO: Crawled 205 pages (at 4 pages/min), scraped 9 items (at 1 items/min)
2026-04-01 06:56:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 241MiB
2026-04-01 06:57:42 [scrapy.extensions.logstats] INFO: Crawled 205 pages (at 0 pages/min), scraped 9 items (at 0 items/min)
2026-04-01 06:57:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 241MiB
2026-04-01 06:58:42 [scrapy.extensions.logstats] INFO: Crawled 205 pages (at 0 pages/min), scraped 9 items (at 0 items/min)
2026-04-01 06:58:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 241MiB
2026-04-01 06:59:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 2394bf60-753c-525e-83e3-2562f363c7b9 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33598/3359721/93408626_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:59:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 2394bf60-753c-525e-83e3-2562f363c7b9 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33598/3359721/93408625_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:59:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item fc4054a6-02e3-542b-98a4-9dc33969f2e6 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33665/3366442/93631622_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:59:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 6dd5cfad-2f25-5699-9d5c-bb1a1dc5b49e due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33631/3363014/93488611_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:59:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 6dd5cfad-2f25-5699-9d5c-bb1a1dc5b49e due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33631/3363014/93488623_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:59:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/citroen-c3-aircross-1-2-e-thp-feel-3366416/details> (referer: None)
2026-04-01 06:59:27 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: a65467ca-78a2-5e1d-a019-d94fba7cb93b with identifier: 3366416
2026-04-01 06:59:27 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a65467ca-78a2-5e1d-a019-d94fba7cb93b with scrape type 1
2026-04-01 06:59:27 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item a65467ca-78a2-5e1d-a019-d94fba7cb93b
2026-04-01 06:59:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366416%252F93630826_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:59:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366416/93630826_hd.jpg not downloaded yet
2026-04-01 06:59:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366416%252F93630829_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:59:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366416/93630829_hd.jpg not downloaded yet
2026-04-01 06:59:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366416%252F93630841_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:59:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366416/93630841_hd.jpg not downloaded yet
2026-04-01 06:59:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366416%252F93630828_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:59:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366416/93630828_hd.jpg not downloaded yet
2026-04-01 06:59:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366416%252F93630843_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:59:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366416/93630843_hd.jpg not downloaded yet
2026-04-01 06:59:28 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=55> (referer: None)
2026-04-01 06:59:28 [autobid.de] INFO: Page 55/145: found 20 items
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366469%27 HTTP/1.1" 200 None
2026-04-01 06:59:28 [autobid.de] INFO: Scrape type for 3366469: 1
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364463%27 HTTP/1.1" 200 None
2026-04-01 06:59:28 [autobid.de] INFO: Scrape type for 3364463: 1
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366475%27 HTTP/1.1" 200 None
2026-04-01 06:59:28 [autobid.de] INFO: Scrape type for 3366475: 1
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366471%27 HTTP/1.1" 200 None
2026-04-01 06:59:28 [autobid.de] INFO: Scrape type for 3366471: 1
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364218%27 HTTP/1.1" 200 None
2026-04-01 06:59:28 [autobid.de] INFO: Scrape type for 3364218: 1
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366418%27 HTTP/1.1" 200 None
2026-04-01 06:59:28 [autobid.de] INFO: Scrape type for 3366418: 1
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366411%27 HTTP/1.1" 200 None
2026-04-01 06:59:28 [autobid.de] INFO: Scrape type for 3366411: 1
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366523%27 HTTP/1.1" 200 None
2026-04-01 06:59:28 [autobid.de] INFO: Scrape type for 3366523: 1
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366457%27 HTTP/1.1" 200 None
2026-04-01 06:59:28 [autobid.de] INFO: Scrape type for 3366457: 1
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366858%27 HTTP/1.1" 200 None
2026-04-01 06:59:28 [autobid.de] INFO: Scrape type for 3366858: 1
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273350065%27 HTTP/1.1" 200 None
2026-04-01 06:59:28 [autobid.de] INFO: Scrape type for 3350065: 1
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359841%27 HTTP/1.1" 200 None
2026-04-01 06:59:28 [autobid.de] INFO: Scrape type for 3359841: 1
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363402%27 HTTP/1.1" 200 None
2026-04-01 06:59:28 [autobid.de] INFO: Scrape type for 3363402: 1
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358936%27 HTTP/1.1" 200 None
2026-04-01 06:59:28 [autobid.de] INFO: Scrape type for 3358936: 1
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366477%27 HTTP/1.1" 200 None
2026-04-01 06:59:28 [autobid.de] INFO: Scrape type for 3366477: 1
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366432%27 HTTP/1.1" 200 None
2026-04-01 06:59:28 [autobid.de] INFO: Scrape type for 3366432: 1
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366478%27 HTTP/1.1" 200 None
2026-04-01 06:59:28 [autobid.de] INFO: Scrape type for 3366478: 1
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357031%27 HTTP/1.1" 200 None
2026-04-01 06:59:28 [autobid.de] INFO: Scrape type for 3357031: 1
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357676%27 HTTP/1.1" 200 None
2026-04-01 06:59:28 [autobid.de] INFO: Scrape type for 3357676: 1
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361721%27 HTTP/1.1" 200 None
2026-04-01 06:59:28 [autobid.de] INFO: Scrape type for 3361721: 1
2026-04-01 06:59:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 6dd5cfad-2f25-5699-9d5c-bb1a1dc5b49e due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33631/3363014/93488625_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:59:28 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/bmw-318d-touring-3366447/details> (referer: None)
2026-04-01 06:59:28 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: ef7da668-dcfb-58fe-89e6-233b6d789f29 with identifier: 3366447
2026-04-01 06:59:28 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ef7da668-dcfb-58fe-89e6-233b6d789f29 with scrape type 1
2026-04-01 06:59:28 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item ef7da668-dcfb-58fe-89e6-233b6d789f29
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366447%252F93631792_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:59:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366447/93631792_hd.jpg not downloaded yet
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366447%252F93631789_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:59:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366447/93631789_hd.jpg not downloaded yet
2026-04-01 06:59:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366447%252F93631800_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:59:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366447/93631800_hd.jpg not downloaded yet
2026-04-01 06:59:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366447%252F93631785_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:59:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366447/93631785_hd.jpg not downloaded yet
2026-04-01 06:59:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366447%252F93631788_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:59:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366447/93631788_hd.jpg not downloaded yet
2026-04-01 06:59:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/skoda-superb-2-0-tdi-dsg-elegance-3366477/details> (referer: None)
2026-04-01 06:59:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 3c126531-3c4c-59f5-9b66-0b899016556a due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33662/3366105/93619551_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:59:30 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 0a3e8d6a-f423-5dca-b3e5-41e7c5f0d6a7 with identifier: 3366477
2026-04-01 06:59:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 0a3e8d6a-f423-5dca-b3e5-41e7c5f0d6a7 with scrape type 1
2026-04-01 06:59:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 0a3e8d6a-f423-5dca-b3e5-41e7c5f0d6a7
2026-04-01 06:59:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366477%252F93633019_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:59:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366477/93633019_hd.jpg not downloaded yet
2026-04-01 06:59:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366477%252F93633014_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:59:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366477/93633014_hd.jpg not downloaded yet
2026-04-01 06:59:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366477%252F93633013_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:59:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366477/93633013_hd.jpg not downloaded yet
2026-04-01 06:59:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366477%252F93633012_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:59:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366477/93633012_hd.jpg not downloaded yet
2026-04-01 06:59:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366477%252F93633023_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 06:59:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366477/93633023_hd.jpg not downloaded yet
2026-04-01 06:59:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 06dabfeb-d0e2-5784-8c3f-3084f111f973 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33665/3366402/93630370_hd.jpg took longer than 180.0 seconds..
2026-04-01 06:59:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=56> (referer: None)
2026-04-01 06:59:33 [autobid.de] INFO: Page 56/145: found 20 items
2026-04-01 06:59:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359835%27 HTTP/1.1" 200 None
2026-04-01 06:59:33 [autobid.de] INFO: Scrape type for 3359835: 1
2026-04-01 06:59:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366421%27 HTTP/1.1" 200 None
2026-04-01 06:59:33 [autobid.de] INFO: Scrape type for 3366421: 1
2026-04-01 06:59:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366424%27 HTTP/1.1" 200 None
2026-04-01 06:59:34 [autobid.de] INFO: Scrape type for 3366424: 1
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366415%27 HTTP/1.1" 200 None
2026-04-01 06:59:34 [autobid.de] INFO: Scrape type for 3366415: 1
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366859%27 HTTP/1.1" 200 None
2026-04-01 06:59:34 [autobid.de] INFO: Scrape type for 3366859: 1
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366444%27 HTTP/1.1" 200 None
2026-04-01 06:59:34 [autobid.de] INFO: Scrape type for 3366444: 1
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366423%27 HTTP/1.1" 200 None
2026-04-01 06:59:34 [autobid.de] INFO: Scrape type for 3366423: 1
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358927%27 HTTP/1.1" 200 None
2026-04-01 06:59:34 [autobid.de] INFO: Scrape type for 3358927: 1
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364980%27 HTTP/1.1" 200 None
2026-04-01 06:59:34 [autobid.de] INFO: Scrape type for 3364980: 1
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366481%27 HTTP/1.1" 200 None
2026-04-01 06:59:34 [autobid.de] INFO: Scrape type for 3366481: 1
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366455%27 HTTP/1.1" 200 None
2026-04-01 06:59:34 [autobid.de] INFO: Scrape type for 3366455: 1
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366459%27 HTTP/1.1" 200 None
2026-04-01 06:59:34 [autobid.de] INFO: Scrape type for 3366459: 1
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353322%27 HTTP/1.1" 200 None
2026-04-01 06:59:34 [autobid.de] INFO: Scrape type for 3353322: 1
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366445%27 HTTP/1.1" 200 None
2026-04-01 06:59:34 [autobid.de] INFO: Scrape type for 3366445: 1
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363404%27 HTTP/1.1" 200 None
2026-04-01 06:59:34 [autobid.de] INFO: Scrape type for 3363404: 1
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358922%27 HTTP/1.1" 200 None
2026-04-01 06:59:34 [autobid.de] INFO: Scrape type for 3358922: 1
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366427%27 HTTP/1.1" 200 None
2026-04-01 06:59:34 [autobid.de] INFO: Scrape type for 3366427: 1
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366443%27 HTTP/1.1" 200 None
2026-04-01 06:59:34 [autobid.de] INFO: Scrape type for 3366443: 1
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366417%27 HTTP/1.1" 200 None
2026-04-01 06:59:34 [autobid.de] INFO: Scrape type for 3366417: 1
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 06:59:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366537%27 HTTP/1.1" 200 None
2026-04-01 06:59:34 [autobid.de] INFO: Scrape type for 3366537: 1
2026-04-01 06:59:42 [scrapy.extensions.logstats] INFO: Crawled 210 pages (at 5 pages/min), scraped 9 items (at 0 items/min)
2026-04-01 06:59:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 242MiB
2026-04-01 07:00:42 [scrapy.extensions.logstats] INFO: Crawled 210 pages (at 0 pages/min), scraped 9 items (at 0 items/min)
2026-04-01 07:00:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 242MiB
2026-04-01 07:01:42 [scrapy.extensions.logstats] INFO: Crawled 210 pages (at 0 pages/min), scraped 9 items (at 0 items/min)
2026-04-01 07:01:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 242MiB
2026-04-01 07:02:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item ccb5e478-91ff-58f3-b091-aa0fc6e3101c due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33617/3361663/93475173_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:02:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 3a019736-3659-5080-b7ff-4c4fc0eee3db due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33440/3343961/92830905_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:02:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ford-transit-kasten-350-l2-2-2-tdci-trend-3366537/details> (referer: None)
2026-04-01 07:02:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: dfb0279c-6fc3-58d6-bf35-798373447ec3 with identifier: 3366537
2026-04-01 07:02:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item dfb0279c-6fc3-58d6-bf35-798373447ec3 with scrape type 1
2026-04-01 07:02:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item dfb0279c-6fc3-58d6-bf35-798373447ec3
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33666%252F3366537%252F93637993_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33666/3366537/93637993_hd.jpg not downloaded yet
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33666%252F3366537%252F93637982_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33666/3366537/93637982_hd.jpg not downloaded yet
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33666%252F3366537%252F93637989_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33666/3366537/93637989_hd.jpg not downloaded yet
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33666%252F3366537%252F93637980_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33666/3366537/93637980_hd.jpg not downloaded yet
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33666%252F3366537%252F93637979_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33666/3366537/93637979_hd.jpg not downloaded yet
2026-04-01 07:02:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=57> (referer: None)
2026-04-01 07:02:22 [autobid.de] INFO: Page 57/145: found 20 items
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366458%27 HTTP/1.1" 200 None
2026-04-01 07:02:22 [autobid.de] INFO: Scrape type for 3366458: 1
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358930%27 HTTP/1.1" 200 None
2026-04-01 07:02:22 [autobid.de] INFO: Scrape type for 3358930: 1
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366547%27 HTTP/1.1" 200 None
2026-04-01 07:02:22 [autobid.de] INFO: Scrape type for 3366547: 1
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365270%27 HTTP/1.1" 200 None
2026-04-01 07:02:22 [autobid.de] INFO: Scrape type for 3365270: 1
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366492%27 HTTP/1.1" 200 None
2026-04-01 07:02:22 [autobid.de] INFO: Scrape type for 3366492: 1
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352520%27 HTTP/1.1" 200 None
2026-04-01 07:02:22 [autobid.de] INFO: Scrape type for 3352520: 1
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352589%27 HTTP/1.1" 200 None
2026-04-01 07:02:22 [autobid.de] INFO: Scrape type for 3352589: 1
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366541%27 HTTP/1.1" 200 None
2026-04-01 07:02:22 [autobid.de] INFO: Scrape type for 3366541: 1
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364219%27 HTTP/1.1" 200 None
2026-04-01 07:02:22 [autobid.de] INFO: Scrape type for 3364219: 1
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359833%27 HTTP/1.1" 200 None
2026-04-01 07:02:22 [autobid.de] INFO: Scrape type for 3359833: 1
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366461%27 HTTP/1.1" 200 None
2026-04-01 07:02:22 [autobid.de] INFO: Scrape type for 3366461: 1
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358933%27 HTTP/1.1" 200 None
2026-04-01 07:02:22 [autobid.de] INFO: Scrape type for 3358933: 1
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366465%27 HTTP/1.1" 200 None
2026-04-01 07:02:22 [autobid.de] INFO: Scrape type for 3366465: 1
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366450%27 HTTP/1.1" 200 None
2026-04-01 07:02:22 [autobid.de] INFO: Scrape type for 3366450: 1
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366542%27 HTTP/1.1" 200 None
2026-04-01 07:02:22 [autobid.de] INFO: Scrape type for 3366542: 1
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361725%27 HTTP/1.1" 200 None
2026-04-01 07:02:22 [autobid.de] INFO: Scrape type for 3361725: 1
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343981%27 HTTP/1.1" 200 None
2026-04-01 07:02:22 [autobid.de] INFO: Scrape type for 3343981: 1
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366534%27 HTTP/1.1" 200 None
2026-04-01 07:02:22 [autobid.de] INFO: Scrape type for 3366534: 1
2026-04-01 07:02:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363406%27 HTTP/1.1" 200 None
2026-04-01 07:02:23 [autobid.de] INFO: Scrape type for 3363406: 1
2026-04-01 07:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358923%27 HTTP/1.1" 200 None
2026-04-01 07:02:23 [autobid.de] INFO: Scrape type for 3358923: 1
2026-04-01 07:02:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-polo-1-2-united-3366417/details> (referer: None)
2026-04-01 07:02:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 097296bb-52e5-5ec9-99a1-0e8013a942e8 with identifier: 3366417
2026-04-01 07:02:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 097296bb-52e5-5ec9-99a1-0e8013a942e8 with scrape type 1
2026-04-01 07:02:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 097296bb-52e5-5ec9-99a1-0e8013a942e8
2026-04-01 07:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366417%252F93630855_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366417/93630855_hd.jpg not downloaded yet
2026-04-01 07:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366417%252F93630861_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366417/93630861_hd.jpg not downloaded yet
2026-04-01 07:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366417%252F93630856_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366417/93630856_hd.jpg not downloaded yet
2026-04-01 07:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366417%252F93630862_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366417/93630862_hd.jpg not downloaded yet
2026-04-01 07:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33665%252F3366417%252F93630850_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33665/3366417/93630850_hd.jpg not downloaded yet
2026-04-01 07:02:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 3a019736-3659-5080-b7ff-4c4fc0eee3db due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33440/3343961/92830919_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:02:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 7325dbfd-c6dd-51bb-937f-40f46218babe due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33590/3358924/93370740_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:02:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=58> (referer: None)
2026-04-01 07:02:26 [autobid.de] INFO: Page 58/145: found 20 items
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366548%27 HTTP/1.1" 200 None
2026-04-01 07:02:26 [autobid.de] INFO: Scrape type for 3366548: 1
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365271%27 HTTP/1.1" 200 None
2026-04-01 07:02:26 [autobid.de] INFO: Scrape type for 3365271: 1
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366493%27 HTTP/1.1" 200 None
2026-04-01 07:02:26 [autobid.de] INFO: Scrape type for 3366493: 1
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353362%27 HTTP/1.1" 200 None
2026-04-01 07:02:26 [autobid.de] INFO: Scrape type for 3353362: 1
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361720%27 HTTP/1.1" 200 None
2026-04-01 07:02:26 [autobid.de] INFO: Scrape type for 3361720: 1
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358831%27 HTTP/1.1" 200 None
2026-04-01 07:02:26 [autobid.de] INFO: Scrape type for 3358831: 1
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366466%27 HTTP/1.1" 200 None
2026-04-01 07:02:26 [autobid.de] INFO: Scrape type for 3366466: 1
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358931%27 HTTP/1.1" 200 None
2026-04-01 07:02:26 [autobid.de] INFO: Scrape type for 3358931: 1
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366468%27 HTTP/1.1" 200 None
2026-04-01 07:02:26 [autobid.de] INFO: Scrape type for 3366468: 1
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366453%27 HTTP/1.1" 200 None
2026-04-01 07:02:26 [autobid.de] INFO: Scrape type for 3366453: 1
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366543%27 HTTP/1.1" 200 None
2026-04-01 07:02:26 [autobid.de] INFO: Scrape type for 3366543: 1
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366454%27 HTTP/1.1" 200 None
2026-04-01 07:02:26 [autobid.de] INFO: Scrape type for 3366454: 1
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343980%27 HTTP/1.1" 200 None
2026-04-01 07:02:26 [autobid.de] INFO: Scrape type for 3343980: 1
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359992%27 HTTP/1.1" 200 None
2026-04-01 07:02:26 [autobid.de] INFO: Scrape type for 3359992: 1
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358920%27 HTTP/1.1" 200 None
2026-04-01 07:02:26 [autobid.de] INFO: Scrape type for 3358920: 1
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366551%27 HTTP/1.1" 200 None
2026-04-01 07:02:26 [autobid.de] INFO: Scrape type for 3366551: 1
2026-04-01 07:02:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365272%27 HTTP/1.1" 200 None
2026-04-01 07:02:27 [autobid.de] INFO: Scrape type for 3365272: 1
2026-04-01 07:02:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366494%27 HTTP/1.1" 200 None
2026-04-01 07:02:27 [autobid.de] INFO: Scrape type for 3366494: 1
2026-04-01 07:02:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273349811%27 HTTP/1.1" 200 None
2026-04-01 07:02:27 [autobid.de] INFO: Scrape type for 3349811: 1
2026-04-01 07:02:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357675%27 HTTP/1.1" 200 None
2026-04-01 07:02:27 [autobid.de] INFO: Scrape type for 3357675: 1
2026-04-01 07:02:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 7325dbfd-c6dd-51bb-937f-40f46218babe due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33590/3358924/93370739_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:02:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 7325dbfd-c6dd-51bb-937f-40f46218babe due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33590/3358924/93370738_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:02:28 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-up-1-0-cup-up-3357675/details> (referer: None)
2026-04-01 07:02:29 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: c411b7a8-f540-5b2c-aabc-98bd6ba03238 with identifier: 3357675
2026-04-01 07:02:29 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item c411b7a8-f540-5b2c-aabc-98bd6ba03238 with scrape type 1
2026-04-01 07:02:29 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item c411b7a8-f540-5b2c-aabc-98bd6ba03238
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33577%252F3357675%252F93488964_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33577/3357675/93488964_hd.jpg not downloaded yet
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33577%252F3357675%252F93488962_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33577/3357675/93488962_hd.jpg not downloaded yet
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33577%252F3357675%252F93488945_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33577/3357675/93488945_hd.jpg not downloaded yet
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33577%252F3357675%252F93488947_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33577/3357675/93488947_hd.jpg not downloaded yet
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33577%252F3357675%252F93488949_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33577/3357675/93488949_hd.jpg not downloaded yet
2026-04-01 07:02:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=59> (referer: None)
2026-04-01 07:02:29 [autobid.de] INFO: Page 59/145: found 20 items
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358943%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [autobid.de] INFO: Scrape type for 3358943: 1
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359990%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [autobid.de] INFO: Scrape type for 3359990: 1
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363408%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [autobid.de] INFO: Scrape type for 3363408: 1
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358929%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [autobid.de] INFO: Scrape type for 3358929: 1
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366476%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [autobid.de] INFO: Scrape type for 3366476: 1
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366456%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [autobid.de] INFO: Scrape type for 3366456: 1
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362882%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [autobid.de] INFO: Scrape type for 3362882: 1
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366544%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [autobid.de] INFO: Scrape type for 3366544: 1
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366675%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [autobid.de] INFO: Scrape type for 3366675: 1
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343945%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [autobid.de] INFO: Scrape type for 3343945: 1
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366533%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [autobid.de] INFO: Scrape type for 3366533: 1
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366473%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [autobid.de] INFO: Scrape type for 3366473: 1
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358926%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [autobid.de] INFO: Scrape type for 3358926: 1
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366549%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [autobid.de] INFO: Scrape type for 3366549: 1
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365273%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [autobid.de] INFO: Scrape type for 3365273: 1
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366495%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [autobid.de] INFO: Scrape type for 3366495: 1
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354956%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [autobid.de] INFO: Scrape type for 3354956: 1
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366570%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [autobid.de] INFO: Scrape type for 3366570: 1
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364220%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [autobid.de] INFO: Scrape type for 3364220: 1
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363211%27 HTTP/1.1" 200 None
2026-04-01 07:02:29 [autobid.de] INFO: Scrape type for 3363211: 1
2026-04-01 07:02:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ford-mondeo-1-6-ecoboost-titanium-3349811/details> (referer: None)
2026-04-01 07:02:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 7325dbfd-c6dd-51bb-937f-40f46218babe due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33590/3358924/93371005_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:02:30 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: ce407514-979f-5dad-bba1-7f523b0ed81b with identifier: 3349811
2026-04-01 07:02:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ce407514-979f-5dad-bba1-7f523b0ed81b with scrape type 1
2026-04-01 07:02:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item ce407514-979f-5dad-bba1-7f523b0ed81b
2026-04-01 07:02:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33499%252F3349811%252F93350614_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33499/3349811/93350614_hd.jpg not downloaded yet
2026-04-01 07:02:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33499%252F3349811%252F93350615_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33499/3349811/93350615_hd.jpg not downloaded yet
2026-04-01 07:02:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33499%252F3349811%252F93350612_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33499/3349811/93350612_hd.jpg not downloaded yet
2026-04-01 07:02:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33499%252F3349811%252F93350613_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33499/3349811/93350613_hd.jpg not downloaded yet
2026-04-01 07:02:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33499%252F3349811%252F93350567_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33499/3349811/93350567_hd.jpg not downloaded yet
2026-04-01 07:02:32 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 7325dbfd-c6dd-51bb-937f-40f46218babe with identifier: 3358924
2026-04-01 07:02:32 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 07:02:32 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=black+Metallic HTTP/1.1" 200 None
2026-04-01 07:02:32 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 07:02:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 07:02:32 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 7325dbfd-c6dd-51bb-937f-40f46218babe with identifier: 3358924
2026-04-01 07:02:32 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: 7325dbfd-c6dd-51bb-937f-40f46218babe with identifier: 3358924 to the API
2026-04-01 07:02:32 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 07:02:32 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 07:02:32 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33590/3358924/93371006_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 80634',
'body_type': 'Saloon',
'category': 'Saloon',
'color': 'black Metallic',
'currency': 'EUR',
'emission_standard': 'EURO 6d-Temp',
'engine': '1.0L',
'engine_horse_power': 101,
'expiration_date': '2026-04-02T08:00:00Z',
'fuel_type': 'Petrol',
'id': '7325dbfd-c6dd-51bb-937f-40f46218babe',
'is_damaged': True,
'is_operable': True,
'is_vat_deductible': False,
'link_to_web_offer': 'https://autobid.de/en/item/ford-fiesta-1-0-ecoboost-automatik-titanium-3358924',
'listing': 'auction',
'location': 'Hamburg',
'make': 'Ford',
'model': 'Fiesta',
'odometer': 121100,
'odometer_reading_unit': 'km',
'origin_country_code': 'DE',
'original_photo_count': 5,
'photo_urls': ['https://cdn.autobid.de/data/cars/33590/3358924/93371006_hd.jpg',
'https://cdn.autobid.de/data/cars/33590/3358924/93371005_hd.jpg',
'https://cdn.autobid.de/data/cars/33590/3358924/93370738_hd.jpg',
'https://cdn.autobid.de/data/cars/33590/3358924/93370739_hd.jpg',
'https://cdn.autobid.de/data/cars/33590/3358924/93370740_hd.jpg'],
'photos': [],
'price': 5500,
'price_includes_vat': False,
'registration_date': '2018-11-01',
'seats_number': 5,
'seller_name': 'Audi Hamburg GmbH',
'title': 'Ford Fiesta 1,0 EcoBoost Automatik "Titanium"',
'transmission': 'automatic',
'vin': None,
'year': 2018}
2026-04-01 07:02:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358924%27 HTTP/1.1" 200 None
2026-04-01 07:02:32 [autobid.de] INFO: Saving data for 3358924: {'created_time': 1775026952.978176, 'last_price_update_time': 1775026952.978186, 'auction_closing_time': 1775116800.0}
2026-04-01 07:02:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3358924') HTTP/1.1" 204 0
2026-04-01 07:02:33 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 7325dbfd-c6dd-51bb-937f-40f46218babe due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33590/3358924/93371006_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:02:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=60> (referer: None)
2026-04-01 07:02:33 [autobid.de] INFO: Page 60/145: found 20 items
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366474%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [autobid.de] INFO: Scrape type for 3366474: 1
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358918%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [autobid.de] INFO: Scrape type for 3358918: 1
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366611%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [autobid.de] INFO: Scrape type for 3366611: 1
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366532%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [autobid.de] INFO: Scrape type for 3366532: 1
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366545%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [autobid.de] INFO: Scrape type for 3366545: 1
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365277%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [autobid.de] INFO: Scrape type for 3365277: 1
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343959%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [autobid.de] INFO: Scrape type for 3343959: 1
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366480%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [autobid.de] INFO: Scrape type for 3366480: 1
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358938%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [autobid.de] INFO: Scrape type for 3358938: 1
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366552%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [autobid.de] INFO: Scrape type for 3366552: 1
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365274%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [autobid.de] INFO: Scrape type for 3365274: 1
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366496%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [autobid.de] INFO: Scrape type for 3366496: 1
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357032%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [autobid.de] INFO: Scrape type for 3357032: 1
2026-04-01 07:02:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-q8-55-tfsi-quattro-tiptronic-3363211/details> (referer: None)
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352591%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [autobid.de] INFO: Scrape type for 3352591: 1
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359839%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [autobid.de] INFO: Scrape type for 3359839: 1
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366601%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [autobid.de] INFO: Scrape type for 3366601: 1
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358935%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [autobid.de] INFO: Scrape type for 3358935: 1
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366472%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [autobid.de] INFO: Scrape type for 3366472: 1
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366546%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [autobid.de] INFO: Scrape type for 3366546: 1
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343978%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [autobid.de] INFO: Scrape type for 3343978: 1
2026-04-01 07:02:33 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 38166911-5772-504e-8b2d-705759cd651e with identifier: 3363211
2026-04-01 07:02:33 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 38166911-5772-504e-8b2d-705759cd651e with scrape type 1
2026-04-01 07:02:33 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 38166911-5772-504e-8b2d-705759cd651e
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33633%252F3363211%252F93499518_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33633/3363211/93499518_hd.jpg not downloaded yet
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33633%252F3363211%252F93499512_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33633/3363211/93499512_hd.jpg not downloaded yet
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33633%252F3363211%252F93499513_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33633/3363211/93499513_hd.jpg not downloaded yet
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33633%252F3363211%252F93499507_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33633/3363211/93499507_hd.jpg not downloaded yet
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33633%252F3363211%252F93499514_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33633/3363211/93499514_hd.jpg not downloaded yet
2026-04-01 07:02:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-passat-variant-2-0-tdi-comfortline-3357032/details> (referer: None)
2026-04-01 07:02:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 2adccbd4-05bc-5683-8363-8f1158c282a2 with identifier: 3357032
2026-04-01 07:02:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 2adccbd4-05bc-5683-8363-8f1158c282a2 with scrape type 1
2026-04-01 07:02:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 2adccbd4-05bc-5683-8363-8f1158c282a2
2026-04-01 07:02:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33571%252F3357032%252F93502116_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33571/3357032/93502116_hd.jpg not downloaded yet
2026-04-01 07:02:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33571%252F3357032%252F93502119_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33571/3357032/93502119_hd.jpg not downloaded yet
2026-04-01 07:02:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33571%252F3357032%252F93502110_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33571/3357032/93502110_hd.jpg not downloaded yet
2026-04-01 07:02:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33571%252F3357032%252F93502113_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33571/3357032/93502113_hd.jpg not downloaded yet
2026-04-01 07:02:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:02:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33571%252F3357032%252F93502095_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:02:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33571/3357032/93502095_hd.jpg not downloaded yet
2026-04-01 07:02:42 [scrapy.extensions.logstats] INFO: Crawled 220 pages (at 10 pages/min), scraped 10 items (at 1 items/min)
2026-04-01 07:02:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 245MiB
2026-04-01 07:03:42 [scrapy.extensions.logstats] INFO: Crawled 220 pages (at 0 pages/min), scraped 10 items (at 0 items/min)
2026-04-01 07:03:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 245MiB
2026-04-01 07:04:42 [scrapy.extensions.logstats] INFO: Crawled 220 pages (at 0 pages/min), scraped 10 items (at 0 items/min)
2026-04-01 07:04:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 245MiB
2026-04-01 07:05:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 3a019736-3659-5080-b7ff-4c4fc0eee3db due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33440/3343961/92830917_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:05:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 3a019736-3659-5080-b7ff-4c4fc0eee3db due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33440/3343961/92830903_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:05:24 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 3a019736-3659-5080-b7ff-4c4fc0eee3db with identifier: 3343961
2026-04-01 07:05:24 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 07:05:24 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=white HTTP/1.1" 200 None
2026-04-01 07:05:24 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 07:05:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 07:05:24 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 3a019736-3659-5080-b7ff-4c4fc0eee3db with identifier: 3343961
2026-04-01 07:05:24 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: 3a019736-3659-5080-b7ff-4c4fc0eee3db with identifier: 3343961 to the API
2026-04-01 07:05:24 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 07:05:24 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 07:05:24 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33440/3343961/92830906_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 80634',
'body_type': 'Flatbed',
'category': 'Flatbed',
'color': 'white',
'currency': 'EUR',
'emission_standard': 'EURO 6d-Temp',
'engine': '2.0L',
'engine_horse_power': 131,
'expiration_date': '2026-04-02T08:00:00Z',
'fuel_type': 'Diesel',
'id': '3a019736-3659-5080-b7ff-4c4fc0eee3db',
'is_damaged': False,
'is_operable': True,
'is_vat_included': True,
'link_to_web_offer': 'https://autobid.de/en/item/ford-transit-doka-pritsche-350-l2-2-0-tdci-trend-3343961',
'listing': 'auction',
'location': 'Bockel',
'make': 'Ford',
'model': 'Transit Doka Pritsche',
'odometer': 66100,
'odometer_reading_unit': 'km',
'origin_country_code': 'DE',
'original_photo_count': 5,
'photo_urls': ['https://cdn.autobid.de/data/cars/33440/3343961/92830906_hd.jpg',
'https://cdn.autobid.de/data/cars/33440/3343961/92830903_hd.jpg',
'https://cdn.autobid.de/data/cars/33440/3343961/92830917_hd.jpg',
'https://cdn.autobid.de/data/cars/33440/3343961/92830919_hd.jpg',
'https://cdn.autobid.de/data/cars/33440/3343961/92830905_hd.jpg'],
'photos': [],
'price': 14900,
'price_includes_vat': True,
'registration_date': '2022-05-01',
'seats_number': 7,
'seller_name': 'Rental company',
'title': 'Ford Transit Doka Pritsche 350 L2 2,0 TDCI "Trend"',
'transmission': 'manual',
'vin': None,
'year': 2022}
2026-04-01 07:05:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:05:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343961%27 HTTP/1.1" 200 None
2026-04-01 07:05:24 [autobid.de] INFO: Saving data for 3343961: {'created_time': 1775027124.826137, 'last_price_update_time': 1775027124.826149, 'auction_closing_time': 1775116800.0}
2026-04-01 07:05:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:05:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3343961') HTTP/1.1" 204 0
2026-04-01 07:05:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 3a019736-3659-5080-b7ff-4c4fc0eee3db due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33440/3343961/92830906_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:05:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item a65467ca-78a2-5e1d-a019-d94fba7cb93b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33665/3366416/93630843_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:05:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item ef7da668-dcfb-58fe-89e6-233b6d789f29 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33665/3366447/93631788_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:05:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item ef7da668-dcfb-58fe-89e6-233b6d789f29 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33665/3366447/93631785_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:05:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 0a3e8d6a-f423-5dca-b3e5-41e7c5f0d6a7 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33665/3366477/93633023_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:05:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 0a3e8d6a-f423-5dca-b3e5-41e7c5f0d6a7 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33665/3366477/93633012_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:05:42 [scrapy.extensions.logstats] INFO: Crawled 220 pages (at 0 pages/min), scraped 11 items (at 1 items/min)
2026-04-01 07:05:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 245MiB
2026-04-01 07:06:42 [scrapy.extensions.logstats] INFO: Crawled 220 pages (at 0 pages/min), scraped 11 items (at 0 items/min)
2026-04-01 07:06:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 245MiB
2026-04-01 07:07:42 [scrapy.extensions.logstats] INFO: Crawled 220 pages (at 0 pages/min), scraped 11 items (at 0 items/min)
2026-04-01 07:07:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 245MiB
2026-04-01 07:08:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item dfb0279c-6fc3-58d6-bf35-798373447ec3 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33666/3366537/93637979_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:08:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 097296bb-52e5-5ec9-99a1-0e8013a942e8 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33665/3366417/93630850_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:08:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=61> (referer: None)
2026-04-01 07:08:22 [autobid.de] INFO: Page 61/145: found 20 items
2026-04-01 07:08:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357033%27 HTTP/1.1" 200 None
2026-04-01 07:08:22 [autobid.de] INFO: Scrape type for 3357033: 1
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354022%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [autobid.de] INFO: Scrape type for 3354022: 1
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365276%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [autobid.de] INFO: Scrape type for 3365276: 1
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343971%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [autobid.de] INFO: Scrape type for 3343971: 1
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366497%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [autobid.de] INFO: Scrape type for 3366497: 1
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343954%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [autobid.de] INFO: Scrape type for 3343954: 1
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366451%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [autobid.de] INFO: Scrape type for 3366451: 1
2026-04-01 07:08:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-crafter-kasten-50-mr-rwd-2-0-tdi-3343978/details> (referer: None)
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354027%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [autobid.de] INFO: Scrape type for 3354027: 1
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354025%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [autobid.de] INFO: Scrape type for 3354025: 1
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364225%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [autobid.de] INFO: Scrape type for 3364225: 1
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352201%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [autobid.de] INFO: Scrape type for 3352201: 1
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359989%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [autobid.de] INFO: Scrape type for 3359989: 1
2026-04-01 07:08:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 6258a5de-ee11-5909-9e5a-6527dae4af30 with identifier: 3343978
2026-04-01 07:08:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6258a5de-ee11-5909-9e5a-6527dae4af30 with scrape type 1
2026-04-01 07:08:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 6258a5de-ee11-5909-9e5a-6527dae4af30
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343978%252F92831708_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343978/92831708_hd.jpg not downloaded yet
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343949%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [autobid.de] INFO: Scrape type for 3343949: 1
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343978%252F92831712_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343978/92831712_hd.jpg not downloaded yet
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343973%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [autobid.de] INFO: Scrape type for 3343973: 1
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343978%252F92831720_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343978/92831720_hd.jpg not downloaded yet
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366499%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [autobid.de] INFO: Scrape type for 3366499: 1
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343978%252F92831719_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343978/92831719_hd.jpg not downloaded yet
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354021%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [autobid.de] INFO: Scrape type for 3354021: 1
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343978%252F92831711_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343978/92831711_hd.jpg not downloaded yet
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354042%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [autobid.de] INFO: Scrape type for 3354042: 1
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358942%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [autobid.de] INFO: Scrape type for 3358942: 1
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354039%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [autobid.de] INFO: Scrape type for 3354039: 1
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343968%27 HTTP/1.1" 200 None
2026-04-01 07:08:23 [autobid.de] INFO: Scrape type for 3343968: 1
2026-04-01 07:08:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 097296bb-52e5-5ec9-99a1-0e8013a942e8 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33665/3366417/93630862_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:08:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ford-transit-custom-320-kasten-2-0-tdci-l2h1-3343954/details> (referer: None)
2026-04-01 07:08:24 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 48dc34ed-d580-5ffd-971e-67604a60d913 with identifier: 3343954
2026-04-01 07:08:24 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 48dc34ed-d580-5ffd-971e-67604a60d913 with scrape type 1
2026-04-01 07:08:24 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 48dc34ed-d580-5ffd-971e-67604a60d913
2026-04-01 07:08:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343954%252F92830462_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343954/92830462_hd.jpg not downloaded yet
2026-04-01 07:08:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343954%252F92830451_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343954/92830451_hd.jpg not downloaded yet
2026-04-01 07:08:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343954%252F92830459_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343954/92830459_hd.jpg not downloaded yet
2026-04-01 07:08:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343954%252F92830465_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343954/92830465_hd.jpg not downloaded yet
2026-04-01 07:08:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343954%252F92830453_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343954/92830453_hd.jpg not downloaded yet
2026-04-01 07:08:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 097296bb-52e5-5ec9-99a1-0e8013a942e8 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33665/3366417/93630856_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:08:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ford-focus-turnier-1-5-tdci-ecoblue-automatik-titanium-x-3343968/details> (referer: None)
2026-04-01 07:08:26 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: ce05e46b-c943-5585-9020-a0b0a9a66242 with identifier: 3343968
2026-04-01 07:08:26 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ce05e46b-c943-5585-9020-a0b0a9a66242 with scrape type 1
2026-04-01 07:08:26 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item ce05e46b-c943-5585-9020-a0b0a9a66242
2026-04-01 07:08:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343968%252F92831112_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343968/92831112_hd.jpg not downloaded yet
2026-04-01 07:08:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343968%252F92831107_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343968/92831107_hd.jpg not downloaded yet
2026-04-01 07:08:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343968%252F92831120_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343968/92831120_hd.jpg not downloaded yet
2026-04-01 07:08:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343968%252F92831111_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343968/92831111_hd.jpg not downloaded yet
2026-04-01 07:08:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343968%252F92831115_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343968/92831115_hd.jpg not downloaded yet
2026-04-01 07:08:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item c411b7a8-f540-5b2c-aabc-98bd6ba03238 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33577/3357675/93488949_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:08:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=62> (referer: None)
2026-04-01 07:08:27 [autobid.de] INFO: Page 62/145: found 20 items
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358262%27 HTTP/1.1" 200 None
2026-04-01 07:08:27 [autobid.de] INFO: Scrape type for 3358262: 1
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366550%27 HTTP/1.1" 200 None
2026-04-01 07:08:27 [autobid.de] INFO: Scrape type for 3366550: 1
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273349279%27 HTTP/1.1" 200 None
2026-04-01 07:08:27 [autobid.de] INFO: Scrape type for 3349279: 1
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354026%27 HTTP/1.1" 200 None
2026-04-01 07:08:27 [autobid.de] INFO: Scrape type for 3354026: 1
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364230%27 HTTP/1.1" 200 None
2026-04-01 07:08:27 [autobid.de] INFO: Scrape type for 3364230: 1
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273349278%27 HTTP/1.1" 200 None
2026-04-01 07:08:27 [autobid.de] INFO: Scrape type for 3349278: 1
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354036%27 HTTP/1.1" 200 None
2026-04-01 07:08:27 [autobid.de] INFO: Scrape type for 3354036: 1
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354037%27 HTTP/1.1" 200 None
2026-04-01 07:08:27 [autobid.de] INFO: Scrape type for 3354037: 1
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366844%27 HTTP/1.1" 200 None
2026-04-01 07:08:27 [autobid.de] INFO: Scrape type for 3366844: 1
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366500%27 HTTP/1.1" 200 None
2026-04-01 07:08:27 [autobid.de] INFO: Scrape type for 3366500: 1
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343955%27 HTTP/1.1" 200 None
2026-04-01 07:08:27 [autobid.de] INFO: Scrape type for 3343955: 1
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343975%27 HTTP/1.1" 200 None
2026-04-01 07:08:27 [autobid.de] INFO: Scrape type for 3343975: 1
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364221%27 HTTP/1.1" 200 None
2026-04-01 07:08:27 [autobid.de] INFO: Scrape type for 3364221: 1
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343964%27 HTTP/1.1" 200 None
2026-04-01 07:08:27 [autobid.de] INFO: Scrape type for 3343964: 1
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343974%27 HTTP/1.1" 200 None
2026-04-01 07:08:27 [autobid.de] INFO: Scrape type for 3343974: 1
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366845%27 HTTP/1.1" 200 None
2026-04-01 07:08:27 [autobid.de] INFO: Scrape type for 3366845: 1
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343962%27 HTTP/1.1" 200 None
2026-04-01 07:08:27 [autobid.de] INFO: Scrape type for 3343962: 1
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354043%27 HTTP/1.1" 200 None
2026-04-01 07:08:27 [autobid.de] INFO: Scrape type for 3354043: 1
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343946%27 HTTP/1.1" 200 None
2026-04-01 07:08:27 [autobid.de] INFO: Scrape type for 3343946: 1
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366847%27 HTTP/1.1" 200 None
2026-04-01 07:08:27 [autobid.de] INFO: Scrape type for 3366847: 1
2026-04-01 07:08:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item ce407514-979f-5dad-bba1-7f523b0ed81b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33499/3349811/93350567_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:08:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item ce407514-979f-5dad-bba1-7f523b0ed81b due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33499/3349811/93350613_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:08:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/peugeot-208-1-2-puretech-allure-3366847/details> (referer: None)
2026-04-01 07:08:30 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: ae2e6cc1-3451-5815-ab2c-a45c6be01793 with identifier: 3366847
2026-04-01 07:08:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ae2e6cc1-3451-5815-ab2c-a45c6be01793 with scrape type 1
2026-04-01 07:08:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item ae2e6cc1-3451-5815-ab2c-a45c6be01793
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33669%252F3366847%252F93643688_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33669/3366847/93643688_hd.jpg not downloaded yet
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33669%252F3366847%252F93643690_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33669/3366847/93643690_hd.jpg not downloaded yet
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33669%252F3366847%252F93643706_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33669/3366847/93643706_hd.jpg not downloaded yet
2026-04-01 07:08:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=63> (referer: None)
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33669%252F3366847%252F93643711_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33669/3366847/93643711_hd.jpg not downloaded yet
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33669%252F3366847%252F93643700_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33669/3366847/93643700_hd.jpg not downloaded yet
2026-04-01 07:08:30 [autobid.de] INFO: Page 63/145: found 20 items
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273344949%27 HTTP/1.1" 200 None
2026-04-01 07:08:30 [autobid.de] INFO: Scrape type for 3344949: 1
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343956%27 HTTP/1.1" 200 None
2026-04-01 07:08:30 [autobid.de] INFO: Scrape type for 3343956: 1
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366846%27 HTTP/1.1" 200 None
2026-04-01 07:08:30 [autobid.de] INFO: Scrape type for 3366846: 1
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366501%27 HTTP/1.1" 200 None
2026-04-01 07:08:30 [autobid.de] INFO: Scrape type for 3366501: 1
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354044%27 HTTP/1.1" 200 None
2026-04-01 07:08:30 [autobid.de] INFO: Scrape type for 3354044: 1
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358944%27 HTTP/1.1" 200 None
2026-04-01 07:08:30 [autobid.de] INFO: Scrape type for 3358944: 1
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354033%27 HTTP/1.1" 200 None
2026-04-01 07:08:30 [autobid.de] INFO: Scrape type for 3354033: 1
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343963%27 HTTP/1.1" 200 None
2026-04-01 07:08:30 [autobid.de] INFO: Scrape type for 3343963: 1
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358946%27 HTTP/1.1" 200 None
2026-04-01 07:08:30 [autobid.de] INFO: Scrape type for 3358946: 1
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273344951%27 HTTP/1.1" 200 None
2026-04-01 07:08:30 [autobid.de] INFO: Scrape type for 3344951: 1
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358941%27 HTTP/1.1" 200 None
2026-04-01 07:08:30 [autobid.de] INFO: Scrape type for 3358941: 1
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366848%27 HTTP/1.1" 200 None
2026-04-01 07:08:30 [autobid.de] INFO: Scrape type for 3366848: 1
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273344953%27 HTTP/1.1" 200 None
2026-04-01 07:08:30 [autobid.de] INFO: Scrape type for 3344953: 1
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364235%27 HTTP/1.1" 200 None
2026-04-01 07:08:31 [autobid.de] INFO: Scrape type for 3364235: 1
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343979%27 HTTP/1.1" 200 None
2026-04-01 07:08:31 [autobid.de] INFO: Scrape type for 3343979: 1
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364229%27 HTTP/1.1" 200 None
2026-04-01 07:08:31 [autobid.de] INFO: Scrape type for 3364229: 1
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343966%27 HTTP/1.1" 200 None
2026-04-01 07:08:31 [autobid.de] INFO: Scrape type for 3343966: 1
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366855%27 HTTP/1.1" 200 None
2026-04-01 07:08:31 [autobid.de] INFO: Scrape type for 3366855: 1
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273350067%27 HTTP/1.1" 200 None
2026-04-01 07:08:31 [autobid.de] INFO: Scrape type for 3350067: 1
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343948%27 HTTP/1.1" 200 None
2026-04-01 07:08:31 [autobid.de] INFO: Scrape type for 3343948: 1
2026-04-01 07:08:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ford-transit-doka-pritsche-310-l2-2-0-tdci-trend-3343946/details> (referer: None)
2026-04-01 07:08:31 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 484a7351-3205-56a2-aa54-ef3ae1a1813a with identifier: 3343946
2026-04-01 07:08:31 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 484a7351-3205-56a2-aa54-ef3ae1a1813a with scrape type 1
2026-04-01 07:08:31 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 484a7351-3205-56a2-aa54-ef3ae1a1813a
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343946%252F92829982_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343946/92829982_hd.jpg not downloaded yet
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343946%252F92829980_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343946/92829980_hd.jpg not downloaded yet
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343946%252F92829972_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343946/92829972_hd.jpg not downloaded yet
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343946%252F92829983_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343946/92829983_hd.jpg not downloaded yet
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343946%252F92829968_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:08:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343946/92829968_hd.jpg not downloaded yet
2026-04-01 07:08:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 38166911-5772-504e-8b2d-705759cd651e due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33633/3363211/93499514_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:08:35 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=64> (referer: None)
2026-04-01 07:08:35 [autobid.de] INFO: Page 64/145: found 20 items
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366849%27 HTTP/1.1" 200 None
2026-04-01 07:08:35 [autobid.de] INFO: Scrape type for 3366849: 1
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366502%27 HTTP/1.1" 200 None
2026-04-01 07:08:35 [autobid.de] INFO: Scrape type for 3366502: 1
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343976%27 HTTP/1.1" 200 None
2026-04-01 07:08:35 [autobid.de] INFO: Scrape type for 3343976: 1
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358257%27 HTTP/1.1" 200 None
2026-04-01 07:08:35 [autobid.de] INFO: Scrape type for 3358257: 1
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343969%27 HTTP/1.1" 200 None
2026-04-01 07:08:35 [autobid.de] INFO: Scrape type for 3343969: 1
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273344950%27 HTTP/1.1" 200 None
2026-04-01 07:08:35 [autobid.de] INFO: Scrape type for 3344950: 1
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361718%27 HTTP/1.1" 200 None
2026-04-01 07:08:35 [autobid.de] INFO: Scrape type for 3361718: 1
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354028%27 HTTP/1.1" 200 None
2026-04-01 07:08:35 [autobid.de] INFO: Scrape type for 3354028: 1
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366850%27 HTTP/1.1" 200 None
2026-04-01 07:08:35 [autobid.de] INFO: Scrape type for 3366850: 1
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273350068%27 HTTP/1.1" 200 None
2026-04-01 07:08:35 [autobid.de] INFO: Scrape type for 3350068: 1
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364234%27 HTTP/1.1" 200 None
2026-04-01 07:08:35 [autobid.de] INFO: Scrape type for 3364234: 1
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343984%27 HTTP/1.1" 200 None
2026-04-01 07:08:35 [autobid.de] INFO: Scrape type for 3343984: 1
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364231%27 HTTP/1.1" 200 None
2026-04-01 07:08:35 [autobid.de] INFO: Scrape type for 3364231: 1
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343944%27 HTTP/1.1" 200 None
2026-04-01 07:08:35 [autobid.de] INFO: Scrape type for 3343944: 1
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343983%27 HTTP/1.1" 200 None
2026-04-01 07:08:35 [autobid.de] INFO: Scrape type for 3343983: 1
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366856%27 HTTP/1.1" 200 None
2026-04-01 07:08:35 [autobid.de] INFO: Scrape type for 3366856: 1
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354032%27 HTTP/1.1" 200 None
2026-04-01 07:08:35 [autobid.de] INFO: Scrape type for 3354032: 1
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366851%27 HTTP/1.1" 200 None
2026-04-01 07:08:35 [autobid.de] INFO: Scrape type for 3366851: 1
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273344954%27 HTTP/1.1" 200 None
2026-04-01 07:08:35 [autobid.de] INFO: Scrape type for 3344954: 1
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:08:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343986%27 HTTP/1.1" 200 None
2026-04-01 07:08:35 [autobid.de] INFO: Scrape type for 3343986: 1
2026-04-01 07:08:42 [scrapy.extensions.logstats] INFO: Crawled 229 pages (at 9 pages/min), scraped 11 items (at 0 items/min)
2026-04-01 07:08:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 246MiB
2026-04-01 07:09:42 [scrapy.extensions.logstats] INFO: Crawled 229 pages (at 0 pages/min), scraped 11 items (at 0 items/min)
2026-04-01 07:09:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 246MiB
2026-04-01 07:10:42 [scrapy.extensions.logstats] INFO: Crawled 229 pages (at 0 pages/min), scraped 11 items (at 0 items/min)
2026-04-01 07:10:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 246MiB
2026-04-01 07:11:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 2adccbd4-05bc-5683-8363-8f1158c282a2 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33571/3357032/93502095_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:11:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 2adccbd4-05bc-5683-8363-8f1158c282a2 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33571/3357032/93502113_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:11:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ford-transit-custom-kombi-320-l2-2-0-ecoblue-trend-3343986/details> (referer: None)
2026-04-01 07:11:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 33844720-0817-5ab6-8fc1-be362e9e0aaa with identifier: 3343986
2026-04-01 07:11:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 33844720-0817-5ab6-8fc1-be362e9e0aaa with scrape type 1
2026-04-01 07:11:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 33844720-0817-5ab6-8fc1-be362e9e0aaa
2026-04-01 07:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343986%252F92831940_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343986/92831940_hd.jpg not downloaded yet
2026-04-01 07:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343986%252F92831935_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343986/92831935_hd.jpg not downloaded yet
2026-04-01 07:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343986%252F92831945_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343986/92831945_hd.jpg not downloaded yet
2026-04-01 07:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343986%252F92831938_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343986/92831938_hd.jpg not downloaded yet
2026-04-01 07:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33440%252F3343986%252F92831950_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33440/3343986/92831950_hd.jpg not downloaded yet
2026-04-01 07:11:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=65> (referer: None)
2026-04-01 07:11:23 [autobid.de] INFO: Page 65/145: found 20 items
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273348021%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [autobid.de] INFO: Scrape type for 3348021: 1
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366852%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [autobid.de] INFO: Scrape type for 3366852: 1
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354034%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [autobid.de] INFO: Scrape type for 3354034: 1
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364233%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [autobid.de] INFO: Scrape type for 3364233: 1
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354045%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [autobid.de] INFO: Scrape type for 3354045: 1
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364228%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [autobid.de] INFO: Scrape type for 3364228: 1
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354024%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [autobid.de] INFO: Scrape type for 3354024: 1
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358832%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [autobid.de] INFO: Scrape type for 3358832: 1
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273344952%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [autobid.de] INFO: Scrape type for 3344952: 1
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354038%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [autobid.de] INFO: Scrape type for 3354038: 1
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366853%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [autobid.de] INFO: Scrape type for 3366853: 1
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354047%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [autobid.de] INFO: Scrape type for 3354047: 1
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354046%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [autobid.de] INFO: Scrape type for 3354046: 1
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273349281%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [autobid.de] INFO: Scrape type for 3349281: 1
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358948%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [autobid.de] INFO: Scrape type for 3358948: 1
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354040%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [autobid.de] INFO: Scrape type for 3354040: 1
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361726%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [autobid.de] INFO: Scrape type for 3361726: 1
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366854%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [autobid.de] INFO: Scrape type for 3366854: 1
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354020%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [autobid.de] INFO: Scrape type for 3354020: 1
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364232%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [autobid.de] INFO: Scrape type for 3364232: 1
2026-04-01 07:11:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ford-transit-kasten-470-2-0-tdci-l4-trend-3344954/details> (referer: None)
2026-04-01 07:11:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: f4f2fd4a-bb2b-5e73-8009-ca19f9d68913 with identifier: 3344954
2026-04-01 07:11:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f4f2fd4a-bb2b-5e73-8009-ca19f9d68913 with scrape type 1
2026-04-01 07:11:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item f4f2fd4a-bb2b-5e73-8009-ca19f9d68913
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33450%252F3344954%252F92887237_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33450/3344954/92887237_hd.jpg not downloaded yet
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33450%252F3344954%252F92887214_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33450/3344954/92887214_hd.jpg not downloaded yet
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33450%252F3344954%252F92887226_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33450/3344954/92887226_hd.jpg not downloaded yet
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33450%252F3344954%252F92887212_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33450/3344954/92887212_hd.jpg not downloaded yet
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33450%252F3344954%252F92887227_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33450/3344954/92887227_hd.jpg not downloaded yet
2026-04-01 07:11:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 2adccbd4-05bc-5683-8363-8f1158c282a2 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33571/3357032/93502110_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:11:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 2adccbd4-05bc-5683-8363-8f1158c282a2 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33571/3357032/93502119_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:11:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=66> (referer: None)
2026-04-01 07:11:26 [autobid.de] INFO: Page 66/145: found 20 items
2026-04-01 07:11:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354030%27 HTTP/1.1" 200 None
2026-04-01 07:11:26 [autobid.de] INFO: Scrape type for 3354030: 1
2026-04-01 07:11:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364224%27 HTTP/1.1" 200 None
2026-04-01 07:11:26 [autobid.de] INFO: Scrape type for 3364224: 1
2026-04-01 07:11:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343977%27 HTTP/1.1" 200 None
2026-04-01 07:11:26 [autobid.de] INFO: Scrape type for 3343977: 1
2026-04-01 07:11:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343950%27 HTTP/1.1" 200 None
2026-04-01 07:11:26 [autobid.de] INFO: Scrape type for 3343950: 1
2026-04-01 07:11:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273349282%27 HTTP/1.1" 200 None
2026-04-01 07:11:26 [autobid.de] INFO: Scrape type for 3349282: 1
2026-04-01 07:11:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354023%27 HTTP/1.1" 200 None
2026-04-01 07:11:26 [autobid.de] INFO: Scrape type for 3354023: 1
2026-04-01 07:11:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357356%27 HTTP/1.1" 200 None
2026-04-01 07:11:26 [autobid.de] INFO: Scrape type for 3357356: 1
2026-04-01 07:11:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343982%27 HTTP/1.1" 200 None
2026-04-01 07:11:26 [autobid.de] INFO: Scrape type for 3343982: 1
2026-04-01 07:11:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358260%27 HTTP/1.1" 200 None
2026-04-01 07:11:26 [autobid.de] INFO: Scrape type for 3358260: 1
2026-04-01 07:11:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343952%27 HTTP/1.1" 200 None
2026-04-01 07:11:26 [autobid.de] INFO: Scrape type for 3343952: 1
2026-04-01 07:11:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358259%27 HTTP/1.1" 200 None
2026-04-01 07:11:26 [autobid.de] INFO: Scrape type for 3358259: 1
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354041%27 HTTP/1.1" 200 None
2026-04-01 07:11:27 [autobid.de] INFO: Scrape type for 3354041: 1
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358945%27 HTTP/1.1" 200 None
2026-04-01 07:11:27 [autobid.de] INFO: Scrape type for 3358945: 1
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343943%27 HTTP/1.1" 200 None
2026-04-01 07:11:27 [autobid.de] INFO: Scrape type for 3343943: 1
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364226%27 HTTP/1.1" 200 None
2026-04-01 07:11:27 [autobid.de] INFO: Scrape type for 3364226: 1
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343965%27 HTTP/1.1" 200 None
2026-04-01 07:11:27 [autobid.de] INFO: Scrape type for 3343965: 1
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364223%27 HTTP/1.1" 200 None
2026-04-01 07:11:27 [autobid.de] INFO: Scrape type for 3364223: 1
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356411%27 HTTP/1.1" 200 None
2026-04-01 07:11:27 [autobid.de] INFO: Scrape type for 3356411: 1
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356410%27 HTTP/1.1" 200 None
2026-04-01 07:11:27 [autobid.de] INFO: Scrape type for 3356410: 1
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354119%27 HTTP/1.1" 200 None
2026-04-01 07:11:27 [autobid.de] INFO: Scrape type for 3354119: 1
2026-04-01 07:11:27 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 2adccbd4-05bc-5683-8363-8f1158c282a2 with identifier: 3357032
2026-04-01 07:11:27 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 2adccbd4-05bc-5683-8363-8f1158c282a2 with identifier: 3357032
2026-04-01 07:11:27 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: 2adccbd4-05bc-5683-8363-8f1158c282a2 with identifier: 3357032 to the API
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 07:11:27 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33571/3357032/93502116_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 80634',
'body_type': 'Estate',
'category': 'Estate',
'color': 'black Pearl effect',
'currency': 'EUR',
'emission_standard': 'EURO 4',
'engine': '2.0L',
'engine_horse_power': 140,
'expiration_date': '2026-04-02T08:00:00Z',
'fuel_type': 'Diesel',
'id': '2adccbd4-05bc-5683-8363-8f1158c282a2',
'is_damaged': False,
'is_operable': True,
'is_vat_deductible': False,
'link_to_web_offer': 'https://autobid.de/en/item/vw-passat-variant-2-0-tdi-comfortline-3357032',
'listing': 'auction',
'location': 'Hamburg',
'make': 'VW',
'model': 'Passat Variant',
'odometer': 271700,
'odometer_reading_unit': 'km',
'origin_country_code': 'DE',
'original_photo_count': 5,
'photo_urls': ['https://cdn.autobid.de/data/cars/33571/3357032/93502116_hd.jpg',
'https://cdn.autobid.de/data/cars/33571/3357032/93502119_hd.jpg',
'https://cdn.autobid.de/data/cars/33571/3357032/93502110_hd.jpg',
'https://cdn.autobid.de/data/cars/33571/3357032/93502113_hd.jpg',
'https://cdn.autobid.de/data/cars/33571/3357032/93502095_hd.jpg'],
'photos': [],
'price': 1000,
'price_includes_vat': False,
'registration_date': '2006-12-01',
'seats_number': 5,
'seller_name': 'Subsidiary/authorised dealer of the brand(s): VW',
'title': 'VW Passat Variant 2,0 TDI "Comfortline"',
'transmission': 'manual',
'vin': None,
'year': 2006}
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357032%27 HTTP/1.1" 200 None
2026-04-01 07:11:27 [autobid.de] INFO: Saving data for 3357032: {'created_time': 1775027487.733058, 'last_price_update_time': 1775027487.733069, 'auction_closing_time': 1775116800.0}
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3357032') HTTP/1.1" 204 0
2026-04-01 07:11:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 2adccbd4-05bc-5683-8363-8f1158c282a2 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33571/3357032/93502116_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:11:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/volvo-ex90-ultra-pure-electric-dual-awd-3364223/details> (referer: None)
2026-04-01 07:11:27 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: f4a21d95-8635-5cbe-af2c-f0cca9a73744 with identifier: 3364223
2026-04-01 07:11:27 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f4a21d95-8635-5cbe-af2c-f0cca9a73744 with scrape type 1
2026-04-01 07:11:27 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item f4a21d95-8635-5cbe-af2c-f0cca9a73744
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33643%252F3364223%252F93587891_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33643/3364223/93587891_hd.jpg not downloaded yet
2026-04-01 07:11:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33643%252F3364223%252F93587893_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33643/3364223/93587893_hd.jpg not downloaded yet
2026-04-01 07:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33643%252F3364223%252F93587895_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33643/3364223/93587895_hd.jpg not downloaded yet
2026-04-01 07:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33643%252F3364223%252F93587897_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33643/3364223/93587897_hd.jpg not downloaded yet
2026-04-01 07:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33643%252F3364223%252F93587899_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:28 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33643/3364223/93587899_hd.jpg not downloaded yet
2026-04-01 07:11:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 38166911-5772-504e-8b2d-705759cd651e due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33633/3363211/93499507_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:11:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 38166911-5772-504e-8b2d-705759cd651e due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33633/3363211/93499513_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:11:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=67> (referer: None)
2026-04-01 07:11:31 [autobid.de] INFO: Page 67/145: found 20 items
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353353%27 HTTP/1.1" 200 None
2026-04-01 07:11:31 [autobid.de] INFO: Scrape type for 3353353: 1
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353715%27 HTTP/1.1" 200 None
2026-04-01 07:11:31 [autobid.de] INFO: Scrape type for 3353715: 1
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353723%27 HTTP/1.1" 200 None
2026-04-01 07:11:31 [autobid.de] INFO: Scrape type for 3353723: 1
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356014%27 HTTP/1.1" 200 None
2026-04-01 07:11:31 [autobid.de] INFO: Scrape type for 3356014: 1
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353722%27 HTTP/1.1" 200 None
2026-04-01 07:11:31 [autobid.de] INFO: Scrape type for 3353722: 1
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353724%27 HTTP/1.1" 200 None
2026-04-01 07:11:31 [autobid.de] INFO: Scrape type for 3353724: 1
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358139%27 HTTP/1.1" 200 None
2026-04-01 07:11:31 [autobid.de] INFO: Scrape type for 3358139: 1
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358841%27 HTTP/1.1" 200 None
2026-04-01 07:11:31 [autobid.de] INFO: Scrape type for 3358841: 1
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353900%27 HTTP/1.1" 200 None
2026-04-01 07:11:31 [autobid.de] INFO: Scrape type for 3353900: 1
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353718%27 HTTP/1.1" 200 None
2026-04-01 07:11:31 [autobid.de] INFO: Scrape type for 3353718: 1
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354882%27 HTTP/1.1" 200 None
2026-04-01 07:11:31 [autobid.de] INFO: Scrape type for 3354882: 1
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356409%27 HTTP/1.1" 200 None
2026-04-01 07:11:31 [autobid.de] INFO: Scrape type for 3356409: 1
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352628%27 HTTP/1.1" 200 None
2026-04-01 07:11:31 [autobid.de] INFO: Scrape type for 3352628: 1
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353719%27 HTTP/1.1" 200 None
2026-04-01 07:11:31 [autobid.de] INFO: Scrape type for 3353719: 1
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354115%27 HTTP/1.1" 200 None
2026-04-01 07:11:31 [autobid.de] INFO: Scrape type for 3354115: 1
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354117%27 HTTP/1.1" 200 None
2026-04-01 07:11:31 [autobid.de] INFO: Scrape type for 3354117: 1
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353716%27 HTTP/1.1" 200 None
2026-04-01 07:11:31 [autobid.de] INFO: Scrape type for 3353716: 1
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353679%27 HTTP/1.1" 200 None
2026-04-01 07:11:31 [autobid.de] INFO: Scrape type for 3353679: 1
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353899%27 HTTP/1.1" 200 None
2026-04-01 07:11:31 [autobid.de] INFO: Scrape type for 3353899: 1
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273341109%27 HTTP/1.1" 200 None
2026-04-01 07:11:31 [autobid.de] INFO: Scrape type for 3341109: 1
2026-04-01 07:11:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 38166911-5772-504e-8b2d-705759cd651e due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33633/3363211/93499512_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:11:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=68> (referer: None)
2026-04-01 07:11:34 [autobid.de] INFO: Page 68/145: found 20 items
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358251%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [autobid.de] INFO: Scrape type for 3358251: 1
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358840%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [autobid.de] INFO: Scrape type for 3358840: 1
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354943%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [autobid.de] INFO: Scrape type for 3354943: 1
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273348995%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [autobid.de] INFO: Scrape type for 3348995: 1
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352630%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [autobid.de] INFO: Scrape type for 3352630: 1
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353680%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [autobid.de] INFO: Scrape type for 3353680: 1
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356013%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [autobid.de] INFO: Scrape type for 3356013: 1
2026-04-01 07:11:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-crafter-35-hochraum-kasten-mr-2-0-tdi-3341109/details> (referer: None)
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354120%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [autobid.de] INFO: Scrape type for 3354120: 1
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353720%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [autobid.de] INFO: Scrape type for 3353720: 1
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354118%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [autobid.de] INFO: Scrape type for 3354118: 1
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353717%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [autobid.de] INFO: Scrape type for 3353717: 1
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354881%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [autobid.de] INFO: Scrape type for 3354881: 1
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356487%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [autobid.de] INFO: Scrape type for 3356487: 1
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359122%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [autobid.de] INFO: Scrape type for 3359122: 1
2026-04-01 07:11:34 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 52338160-ebd0-5769-9d63-e7fa7dc6bbf4 with identifier: 3341109
2026-04-01 07:11:34 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 52338160-ebd0-5769-9d63-e7fa7dc6bbf4 with scrape type 1
2026-04-01 07:11:34 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 52338160-ebd0-5769-9d63-e7fa7dc6bbf4
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33412%252F3341109%252F93387211_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33412/3341109/93387211_hd.jpg not downloaded yet
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356902%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [autobid.de] INFO: Scrape type for 3356902: 1
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33412%252F3341109%252F93387213_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33412/3341109/93387213_hd.jpg not downloaded yet
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354941%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [autobid.de] INFO: Scrape type for 3354941: 1
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33412%252F3341109%252F93387219_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33412/3341109/93387219_hd.jpg not downloaded yet
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343594%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [autobid.de] INFO: Scrape type for 3343594: 1
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33412%252F3341109%252F93387215_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33412/3341109/93387215_hd.jpg not downloaded yet
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353901%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [autobid.de] INFO: Scrape type for 3353901: 1
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33412%252F3341109%252F93387210_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33412/3341109/93387210_hd.jpg not downloaded yet
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354121%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [autobid.de] INFO: Scrape type for 3354121: 1
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354116%27 HTTP/1.1" 200 None
2026-04-01 07:11:34 [autobid.de] INFO: Scrape type for 3354116: 1
2026-04-01 07:11:35 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/opel-corsa-1-3-cdti-150-jahre-opel-3356013/details> (referer: None)
2026-04-01 07:11:35 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 0a4a2cd1-2ed6-5a84-83f3-1d4d94af97d5 with identifier: 3356013
2026-04-01 07:11:35 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 0a4a2cd1-2ed6-5a84-83f3-1d4d94af97d5 with scrape type 1
2026-04-01 07:11:35 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 0a4a2cd1-2ed6-5a84-83f3-1d4d94af97d5
2026-04-01 07:11:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33561%252F3356013%252F93368903_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33561/3356013/93368903_hd.jpg not downloaded yet
2026-04-01 07:11:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33561%252F3356013%252F93368905_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33561/3356013/93368905_hd.jpg not downloaded yet
2026-04-01 07:11:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33561%252F3356013%252F93368898_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33561/3356013/93368898_hd.jpg not downloaded yet
2026-04-01 07:11:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33561%252F3356013%252F93368896_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33561/3356013/93368896_hd.jpg not downloaded yet
2026-04-01 07:11:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33561%252F3356013%252F93368906_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:11:35 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33561/3356013/93368906_hd.jpg not downloaded yet
2026-04-01 07:11:39 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=69> (referer: None)
2026-04-01 07:11:39 [autobid.de] INFO: Page 69/145: found 20 items
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353354%27 HTTP/1.1" 200 None
2026-04-01 07:11:39 [autobid.de] INFO: Scrape type for 3353354: 1
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358114%27 HTTP/1.1" 200 None
2026-04-01 07:11:39 [autobid.de] INFO: Scrape type for 3358114: 1
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358851%27 HTTP/1.1" 200 None
2026-04-01 07:11:39 [autobid.de] INFO: Scrape type for 3358851: 1
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356899%27 HTTP/1.1" 200 None
2026-04-01 07:11:39 [autobid.de] INFO: Scrape type for 3356899: 1
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354944%27 HTTP/1.1" 200 None
2026-04-01 07:11:39 [autobid.de] INFO: Scrape type for 3354944: 1
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356903%27 HTTP/1.1" 200 None
2026-04-01 07:11:39 [autobid.de] INFO: Scrape type for 3356903: 1
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354883%27 HTTP/1.1" 200 None
2026-04-01 07:11:39 [autobid.de] INFO: Scrape type for 3354883: 1
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356848%27 HTTP/1.1" 200 None
2026-04-01 07:11:39 [autobid.de] INFO: Scrape type for 3356848: 1
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356900%27 HTTP/1.1" 200 None
2026-04-01 07:11:39 [autobid.de] INFO: Scrape type for 3356900: 1
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359123%27 HTTP/1.1" 200 None
2026-04-01 07:11:39 [autobid.de] INFO: Scrape type for 3359123: 1
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273349118%27 HTTP/1.1" 200 None
2026-04-01 07:11:39 [autobid.de] INFO: Scrape type for 3349118: 1
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343287%27 HTTP/1.1" 200 None
2026-04-01 07:11:39 [autobid.de] INFO: Scrape type for 3343287: 1
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354942%27 HTTP/1.1" 200 None
2026-04-01 07:11:39 [autobid.de] INFO: Scrape type for 3354942: 1
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356846%27 HTTP/1.1" 200 None
2026-04-01 07:11:39 [autobid.de] INFO: Scrape type for 3356846: 1
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356901%27 HTTP/1.1" 200 None
2026-04-01 07:11:39 [autobid.de] INFO: Scrape type for 3356901: 1
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356847%27 HTTP/1.1" 200 None
2026-04-01 07:11:39 [autobid.de] INFO: Scrape type for 3356847: 1
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354940%27 HTTP/1.1" 200 None
2026-04-01 07:11:39 [autobid.de] INFO: Scrape type for 3354940: 1
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359124%27 HTTP/1.1" 200 None
2026-04-01 07:11:39 [autobid.de] INFO: Scrape type for 3359124: 1
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358181%27 HTTP/1.1" 200 None
2026-04-01 07:11:39 [autobid.de] INFO: Scrape type for 3358181: 1
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:11:39 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356886%27 HTTP/1.1" 200 None
2026-04-01 07:11:39 [autobid.de] INFO: Scrape type for 3356886: 1
2026-04-01 07:11:42 [scrapy.extensions.logstats] INFO: Crawled 239 pages (at 10 pages/min), scraped 12 items (at 1 items/min)
2026-04-01 07:11:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 250MiB
2026-04-01 07:12:42 [scrapy.extensions.logstats] INFO: Crawled 239 pages (at 0 pages/min), scraped 12 items (at 0 items/min)
2026-04-01 07:12:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 250MiB
2026-04-01 07:13:42 [scrapy.extensions.logstats] INFO: Crawled 239 pages (at 0 pages/min), scraped 12 items (at 0 items/min)
2026-04-01 07:13:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 250MiB
2026-04-01 07:14:20 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 38166911-5772-504e-8b2d-705759cd651e with identifier: 3363211
2026-04-01 07:14:20 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 07:14:20 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=Benzin+Mildhybrid HTTP/1.1" 200 None
2026-04-01 07:14:21 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 07:14:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 07:14:21 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 38166911-5772-504e-8b2d-705759cd651e with identifier: 3363211
2026-04-01 07:14:21 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: 38166911-5772-504e-8b2d-705759cd651e with identifier: 3363211 to the API
2026-04-01 07:14:21 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 07:14:21 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 07:14:21 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33633/3363211/93499518_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 80634',
'body_type': 'All-terrain vehicle',
'category': 'All-terrain vehicle',
'color': 'black Metallic',
'currency': 'EUR',
'emission_standard': 'EURO 6d',
'engine': '3.0L',
'engine_horse_power': 340,
'expiration_date': '2026-04-02T08:00:00Z',
'fuel_type': 'Petrol Mild Hybrid',
'id': '38166911-5772-504e-8b2d-705759cd651e',
'is_damaged': False,
'is_operable': True,
'is_vat_included': True,
'link_to_web_offer': 'https://autobid.de/en/item/audi-q8-55-tfsi-quattro-tiptronic-3363211',
'listing': 'auction',
'location': 'Hamburg',
'make': 'Audi',
'model': 'Q8',
'odometer': 54700,
'odometer_reading_unit': 'km',
'origin_country_code': 'DE',
'original_photo_count': 5,
'photo_urls': ['https://cdn.autobid.de/data/cars/33633/3363211/93499518_hd.jpg',
'https://cdn.autobid.de/data/cars/33633/3363211/93499512_hd.jpg',
'https://cdn.autobid.de/data/cars/33633/3363211/93499513_hd.jpg',
'https://cdn.autobid.de/data/cars/33633/3363211/93499507_hd.jpg',
'https://cdn.autobid.de/data/cars/33633/3363211/93499514_hd.jpg'],
'photos': [],
'price': 33200,
'price_includes_vat': True,
'registration_date': '2023-02-01',
'seats_number': 5,
'seller_name': None,
'title': 'Audi Q8 55 TFSI quattro Tiptronic',
'transmission': 'automatic',
'vin': None,
'year': 2023}
2026-04-01 07:14:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363211%27 HTTP/1.1" 200 None
2026-04-01 07:14:21 [autobid.de] INFO: Saving data for 3363211: {'created_time': 1775027661.516796, 'last_price_update_time': 1775027661.516809, 'auction_closing_time': 1775116800.0}
2026-04-01 07:14:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3363211') HTTP/1.1" 204 0
2026-04-01 07:14:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 38166911-5772-504e-8b2d-705759cd651e due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33633/3363211/93499518_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:14:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 48dc34ed-d580-5ffd-971e-67604a60d913 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33440/3343954/92830453_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:14:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/skoda-yeti-2-0-tdi-4x4-3356886/details> (referer: None)
2026-04-01 07:14:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: d7cab2d4-b8e3-5113-9d8b-707e3a8ddde5 with identifier: 3356886
2026-04-01 07:14:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item d7cab2d4-b8e3-5113-9d8b-707e3a8ddde5 with scrape type 1
2026-04-01 07:14:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item d7cab2d4-b8e3-5113-9d8b-707e3a8ddde5
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33569%252F3356886%252F93376968_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:14:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33569/3356886/93376968_hd.jpg not downloaded yet
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33569%252F3356886%252F93376979_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:14:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33569/3356886/93376979_hd.jpg not downloaded yet
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33569%252F3356886%252F93377028_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:14:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33569/3356886/93377028_hd.jpg not downloaded yet
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33569%252F3356886%252F93376997_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:14:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33569/3356886/93376997_hd.jpg not downloaded yet
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33569%252F3356886%252F93376985_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:14:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33569/3356886/93376985_hd.jpg not downloaded yet
2026-04-01 07:14:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=70> (referer: None)
2026-04-01 07:14:22 [autobid.de] INFO: Page 70/145: found 20 items
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359125%27 HTTP/1.1" 200 None
2026-04-01 07:14:22 [autobid.de] INFO: Scrape type for 3359125: 1
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356887%27 HTTP/1.1" 200 None
2026-04-01 07:14:22 [autobid.de] INFO: Scrape type for 3356887: 1
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359900%27 HTTP/1.1" 200 None
2026-04-01 07:14:22 [autobid.de] INFO: Scrape type for 3359900: 1
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356485%27 HTTP/1.1" 200 None
2026-04-01 07:14:22 [autobid.de] INFO: Scrape type for 3356485: 1
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356888%27 HTTP/1.1" 200 None
2026-04-01 07:14:22 [autobid.de] INFO: Scrape type for 3356888: 1
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356486%27 HTTP/1.1" 200 None
2026-04-01 07:14:22 [autobid.de] INFO: Scrape type for 3356486: 1
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356484%27 HTTP/1.1" 200 None
2026-04-01 07:14:22 [autobid.de] INFO: Scrape type for 3356484: 1
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356890%27 HTTP/1.1" 200 None
2026-04-01 07:14:22 [autobid.de] INFO: Scrape type for 3356890: 1
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357515%27 HTTP/1.1" 200 None
2026-04-01 07:14:22 [autobid.de] INFO: Scrape type for 3357515: 1
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359901%27 HTTP/1.1" 200 None
2026-04-01 07:14:22 [autobid.de] INFO: Scrape type for 3359901: 1
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273309380%27 HTTP/1.1" 200 None
2026-04-01 07:14:22 [autobid.de] INFO: Scrape type for 3309380: 1
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359082%27 HTTP/1.1" 200 None
2026-04-01 07:14:22 [autobid.de] INFO: Scrape type for 3359082: 1
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358837%27 HTTP/1.1" 200 None
2026-04-01 07:14:22 [autobid.de] INFO: Scrape type for 3358837: 1
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358839%27 HTTP/1.1" 200 None
2026-04-01 07:14:22 [autobid.de] INFO: Scrape type for 3358839: 1
2026-04-01 07:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358854%27 HTTP/1.1" 200 None
2026-04-01 07:14:23 [autobid.de] INFO: Scrape type for 3358854: 1
2026-04-01 07:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359902%27 HTTP/1.1" 200 None
2026-04-01 07:14:23 [autobid.de] INFO: Scrape type for 3359902: 1
2026-04-01 07:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354910%27 HTTP/1.1" 200 None
2026-04-01 07:14:23 [autobid.de] INFO: Scrape type for 3354910: 1
2026-04-01 07:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358849%27 HTTP/1.1" 200 None
2026-04-01 07:14:23 [autobid.de] INFO: Scrape type for 3358849: 1
2026-04-01 07:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358853%27 HTTP/1.1" 200 None
2026-04-01 07:14:23 [autobid.de] INFO: Scrape type for 3358853: 1
2026-04-01 07:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358836%27 HTTP/1.1" 200 None
2026-04-01 07:14:23 [autobid.de] INFO: Scrape type for 3358836: 1
2026-04-01 07:14:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-golf-plus-1-4-tsi-comfortline-3358181/details> (referer: None)
2026-04-01 07:14:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 1f581311-3a1c-54ed-801a-d65d6dec32b7 with identifier: 3358181
2026-04-01 07:14:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 1f581311-3a1c-54ed-801a-d65d6dec32b7 with scrape type 1
2026-04-01 07:14:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 1f581311-3a1c-54ed-801a-d65d6dec32b7
2026-04-01 07:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33582%252F3358181%252F93376808_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:14:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33582/3358181/93376808_hd.jpg not downloaded yet
2026-04-01 07:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33582%252F3358181%252F93376821_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:14:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33582/3358181/93376821_hd.jpg not downloaded yet
2026-04-01 07:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33582%252F3358181%252F93376802_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:14:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33582/3358181/93376802_hd.jpg not downloaded yet
2026-04-01 07:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33582%252F3358181%252F93376792_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:14:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33582/3358181/93376792_hd.jpg not downloaded yet
2026-04-01 07:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33582%252F3358181%252F93376829_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:14:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33582/3358181/93376829_hd.jpg not downloaded yet
2026-04-01 07:14:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 48dc34ed-d580-5ffd-971e-67604a60d913 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33440/3343954/92830465_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:14:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item ce05e46b-c943-5585-9020-a0b0a9a66242 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33440/3343968/92831115_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:14:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item ce05e46b-c943-5585-9020-a0b0a9a66242 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33440/3343968/92831111_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:14:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=71> (referer: None)
2026-04-01 07:14:27 [autobid.de] INFO: Page 71/145: found 20 items
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358835%27 HTTP/1.1" 200 None
2026-04-01 07:14:27 [autobid.de] INFO: Scrape type for 3358835: 1
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358850%27 HTTP/1.1" 200 None
2026-04-01 07:14:27 [autobid.de] INFO: Scrape type for 3358850: 1
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358838%27 HTTP/1.1" 200 None
2026-04-01 07:14:27 [autobid.de] INFO: Scrape type for 3358838: 1
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354909%27 HTTP/1.1" 200 None
2026-04-01 07:14:27 [autobid.de] INFO: Scrape type for 3354909: 1
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358852%27 HTTP/1.1" 200 None
2026-04-01 07:14:27 [autobid.de] INFO: Scrape type for 3358852: 1
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359903%27 HTTP/1.1" 200 None
2026-04-01 07:14:27 [autobid.de] INFO: Scrape type for 3359903: 1
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359904%27 HTTP/1.1" 200 None
2026-04-01 07:14:27 [autobid.de] INFO: Scrape type for 3359904: 1
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360535%27 HTTP/1.1" 200 None
2026-04-01 07:14:27 [autobid.de] INFO: Scrape type for 3360535: 1
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361102%27 HTTP/1.1" 200 None
2026-04-01 07:14:27 [autobid.de] INFO: Scrape type for 3361102: 1
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361103%27 HTTP/1.1" 200 None
2026-04-01 07:14:27 [autobid.de] INFO: Scrape type for 3361103: 1
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361104%27 HTTP/1.1" 200 None
2026-04-01 07:14:27 [autobid.de] INFO: Scrape type for 3361104: 1
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361105%27 HTTP/1.1" 200 None
2026-04-01 07:14:27 [autobid.de] INFO: Scrape type for 3361105: 1
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273321906%27 HTTP/1.1" 200 None
2026-04-01 07:14:27 [autobid.de] INFO: Scrape type for 3321906: 1
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361991%27 HTTP/1.1" 200 None
2026-04-01 07:14:27 [autobid.de] INFO: Scrape type for 3361991: 1
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358269%27 HTTP/1.1" 200 None
2026-04-01 07:14:27 [autobid.de] INFO: Scrape type for 3358269: 1
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361992%27 HTTP/1.1" 200 None
2026-04-01 07:14:27 [autobid.de] INFO: Scrape type for 3361992: 1
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273350600%27 HTTP/1.1" 200 None
2026-04-01 07:14:27 [autobid.de] INFO: Scrape type for 3350600: 1
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362007%27 HTTP/1.1" 200 None
2026-04-01 07:14:27 [autobid.de] INFO: Scrape type for 3362007: 1
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362008%27 HTTP/1.1" 200 None
2026-04-01 07:14:27 [autobid.de] INFO: Scrape type for 3362008: 1
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273344344%27 HTTP/1.1" 200 None
2026-04-01 07:14:27 [autobid.de] INFO: Scrape type for 3344344: 1
2026-04-01 07:14:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item ae2e6cc1-3451-5815-ab2c-a45c6be01793 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33669/3366847/93643690_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:14:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 484a7351-3205-56a2-aa54-ef3ae1a1813a due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33440/3343946/92829968_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:14:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-passat-alltrack-2-0-tdi-bmt-4motion-3344344/details> (referer: None)
2026-04-01 07:14:30 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: edd97ecd-534d-5a65-ad1a-b284140d0e18 with identifier: 3344344
2026-04-01 07:14:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item edd97ecd-534d-5a65-ad1a-b284140d0e18 with scrape type 1
2026-04-01 07:14:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item edd97ecd-534d-5a65-ad1a-b284140d0e18
2026-04-01 07:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33444%252F3344344%252F93095336_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:14:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33444/3344344/93095336_hd.jpg not downloaded yet
2026-04-01 07:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33444%252F3344344%252F93095339_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:14:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33444/3344344/93095339_hd.jpg not downloaded yet
2026-04-01 07:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33444%252F3344344%252F93095349_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:14:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33444/3344344/93095349_hd.jpg not downloaded yet
2026-04-01 07:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33444%252F3344344%252F93095346_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:14:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33444/3344344/93095346_hd.jpg not downloaded yet
2026-04-01 07:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33444%252F3344344%252F93095342_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:14:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33444/3344344/93095342_hd.jpg not downloaded yet
2026-04-01 07:14:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=72> (referer: None)
2026-04-01 07:14:31 [autobid.de] INFO: Page 72/145: found 20 items
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362009%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [autobid.de] INFO: Scrape type for 3362009: 1
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358273%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [autobid.de] INFO: Scrape type for 3358273: 1
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273344348%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [autobid.de] INFO: Scrape type for 3344348: 1
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362823%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [autobid.de] INFO: Scrape type for 3362823: 1
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273344375%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [autobid.de] INFO: Scrape type for 3344375: 1
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359045%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [autobid.de] INFO: Scrape type for 3359045: 1
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360511%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [autobid.de] INFO: Scrape type for 3360511: 1
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358272%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [autobid.de] INFO: Scrape type for 3358272: 1
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361996%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [autobid.de] INFO: Scrape type for 3361996: 1
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354472%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [autobid.de] INFO: Scrape type for 3354472: 1
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360494%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [autobid.de] INFO: Scrape type for 3360494: 1
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363262%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [autobid.de] INFO: Scrape type for 3363262: 1
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358281%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [autobid.de] INFO: Scrape type for 3358281: 1
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273345666%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [autobid.de] INFO: Scrape type for 3345666: 1
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361997%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [autobid.de] INFO: Scrape type for 3361997: 1
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273344343%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [autobid.de] INFO: Scrape type for 3344343: 1
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353270%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [autobid.de] INFO: Scrape type for 3353270: 1
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353842%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [autobid.de] INFO: Scrape type for 3353842: 1
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360097%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [autobid.de] INFO: Scrape type for 3360097: 1
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357372%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [autobid.de] INFO: Scrape type for 3357372: 1
2026-04-01 07:14:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/bmw-x3-2-0d-3362008/details> (referer: None)
2026-04-01 07:14:31 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 748d7a6e-0175-5a99-876e-22e7c46a2ad0 with identifier: 3362008
2026-04-01 07:14:31 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 748d7a6e-0175-5a99-876e-22e7c46a2ad0 with scrape type 1
2026-04-01 07:14:31 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 748d7a6e-0175-5a99-876e-22e7c46a2ad0
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33621%252F3362008%252F93459841_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33621/3362008/93459841_hd.jpg not downloaded yet
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33621%252F3362008%252F93459840_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33621/3362008/93459840_hd.jpg not downloaded yet
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33621%252F3362008%252F93459856_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33621/3362008/93459856_hd.jpg not downloaded yet
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33621%252F3362008%252F93459839_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33621/3362008/93459839_hd.jpg not downloaded yet
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33621%252F3362008%252F93459838_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:14:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33621/3362008/93459838_hd.jpg not downloaded yet
2026-04-01 07:14:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 484a7351-3205-56a2-aa54-ef3ae1a1813a due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33440/3343946/92829983_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:14:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=73> (referer: None)
2026-04-01 07:14:34 [autobid.de] INFO: Page 73/145: found 20 items
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362825%27 HTTP/1.1" 200 None
2026-04-01 07:14:34 [autobid.de] INFO: Scrape type for 3362825: 1
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358288%27 HTTP/1.1" 200 None
2026-04-01 07:14:34 [autobid.de] INFO: Scrape type for 3358288: 1
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361423%27 HTTP/1.1" 200 None
2026-04-01 07:14:34 [autobid.de] INFO: Scrape type for 3361423: 1
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362924%27 HTTP/1.1" 200 None
2026-04-01 07:14:34 [autobid.de] INFO: Scrape type for 3362924: 1
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363257%27 HTTP/1.1" 200 None
2026-04-01 07:14:34 [autobid.de] INFO: Scrape type for 3363257: 1
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353841%27 HTTP/1.1" 200 None
2026-04-01 07:14:34 [autobid.de] INFO: Scrape type for 3353841: 1
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360116%27 HTTP/1.1" 200 None
2026-04-01 07:14:34 [autobid.de] INFO: Scrape type for 3360116: 1
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360111%27 HTTP/1.1" 200 None
2026-04-01 07:14:34 [autobid.de] INFO: Scrape type for 3360111: 1
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353843%27 HTTP/1.1" 200 None
2026-04-01 07:14:34 [autobid.de] INFO: Scrape type for 3353843: 1
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360548%27 HTTP/1.1" 200 None
2026-04-01 07:14:34 [autobid.de] INFO: Scrape type for 3360548: 1
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361953%27 HTTP/1.1" 200 None
2026-04-01 07:14:34 [autobid.de] INFO: Scrape type for 3361953: 1
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363258%27 HTTP/1.1" 200 None
2026-04-01 07:14:34 [autobid.de] INFO: Scrape type for 3363258: 1
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353844%27 HTTP/1.1" 200 None
2026-04-01 07:14:34 [autobid.de] INFO: Scrape type for 3353844: 1
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363259%27 HTTP/1.1" 200 None
2026-04-01 07:14:34 [autobid.de] INFO: Scrape type for 3363259: 1
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357693%27 HTTP/1.1" 200 None
2026-04-01 07:14:34 [autobid.de] INFO: Scrape type for 3357693: 1
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363294%27 HTTP/1.1" 200 None
2026-04-01 07:14:34 [autobid.de] INFO: Scrape type for 3363294: 1
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360610%27 HTTP/1.1" 200 None
2026-04-01 07:14:34 [autobid.de] INFO: Scrape type for 3360610: 1
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353845%27 HTTP/1.1" 200 None
2026-04-01 07:14:34 [autobid.de] INFO: Scrape type for 3353845: 1
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362002%27 HTTP/1.1" 200 None
2026-04-01 07:14:34 [autobid.de] INFO: Scrape type for 3362002: 1
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:14:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359136%27 HTTP/1.1" 200 None
2026-04-01 07:14:34 [autobid.de] INFO: Scrape type for 3359136: 1
2026-04-01 07:14:42 [scrapy.extensions.logstats] INFO: Crawled 247 pages (at 8 pages/min), scraped 13 items (at 1 items/min)
2026-04-01 07:14:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 250MiB
2026-04-01 07:15:42 [scrapy.extensions.logstats] INFO: Crawled 247 pages (at 0 pages/min), scraped 13 items (at 0 items/min)
2026-04-01 07:15:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 250MiB
2026-04-01 07:16:42 [scrapy.extensions.logstats] INFO: Crawled 247 pages (at 0 pages/min), scraped 13 items (at 0 items/min)
2026-04-01 07:16:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 250MiB
2026-04-01 07:17:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 33844720-0817-5ab6-8fc1-be362e9e0aaa due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33440/3343986/92831950_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:17:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item f4f2fd4a-bb2b-5e73-8009-ca19f9d68913 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33450/3344954/92887227_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:17:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=74> (referer: None)
2026-04-01 07:17:22 [autobid.de] INFO: Page 74/145: found 20 items
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363203%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [autobid.de] INFO: Scrape type for 3363203: 1
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362005%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [autobid.de] INFO: Scrape type for 3362005: 1
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360417%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [autobid.de] INFO: Scrape type for 3360417: 1
2026-04-01 07:17:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/skoda-yeti-2-0-tdi-4x4-ambition-3359136/details> (referer: None)
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360479%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [autobid.de] INFO: Scrape type for 3360479: 1
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353846%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [autobid.de] INFO: Scrape type for 3353846: 1
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360952%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [autobid.de] INFO: Scrape type for 3360952: 1
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273330899%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [autobid.de] INFO: Scrape type for 3330899: 1
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273336729%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [autobid.de] INFO: Scrape type for 3336729: 1
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359058%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [autobid.de] INFO: Scrape type for 3359058: 1
2026-04-01 07:17:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 22e0b0d1-ddc8-58e0-a523-c314b657d6a5 with identifier: 3359136
2026-04-01 07:17:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 22e0b0d1-ddc8-58e0-a523-c314b657d6a5 with scrape type 1
2026-04-01 07:17:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 22e0b0d1-ddc8-58e0-a523-c314b657d6a5
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33592%252F3359136%252F93358364_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33592/3359136/93358364_hd.jpg not downloaded yet
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273344354%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [autobid.de] INFO: Scrape type for 3344354: 1
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33592%252F3359136%252F93358373_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33592/3359136/93358373_hd.jpg not downloaded yet
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362822%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [autobid.de] INFO: Scrape type for 3362822: 1
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33592%252F3359136%252F93358371_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33592/3359136/93358371_hd.jpg not downloaded yet
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363208%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [autobid.de] INFO: Scrape type for 3363208: 1
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33592%252F3359136%252F93358363_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33592/3359136/93358363_hd.jpg not downloaded yet
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363241%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [autobid.de] INFO: Scrape type for 3363241: 1
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33592%252F3359136%252F93358355_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33592/3359136/93358355_hd.jpg not downloaded yet
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353847%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [autobid.de] INFO: Scrape type for 3353847: 1
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362006%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [autobid.de] INFO: Scrape type for 3362006: 1
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365573%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [autobid.de] INFO: Scrape type for 3365573: 1
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365574%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [autobid.de] INFO: Scrape type for 3365574: 1
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365575%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [autobid.de] INFO: Scrape type for 3365575: 1
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362000%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [autobid.de] INFO: Scrape type for 3362000: 1
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361993%27 HTTP/1.1" 200 None
2026-04-01 07:17:22 [autobid.de] INFO: Scrape type for 3361993: 1
2026-04-01 07:17:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-golf-gti-clubsport-dsg-3362002/details> (referer: None)
2026-04-01 07:17:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: e927ffea-0ffc-59c1-9ddc-84698edea946 with identifier: 3362002
2026-04-01 07:17:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e927ffea-0ffc-59c1-9ddc-84698edea946 with scrape type 1
2026-04-01 07:17:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item e927ffea-0ffc-59c1-9ddc-84698edea946
2026-04-01 07:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33621%252F3362002%252F93459622_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33621/3362002/93459622_hd.jpg not downloaded yet
2026-04-01 07:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33621%252F3362002%252F93459606_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33621/3362002/93459606_hd.jpg not downloaded yet
2026-04-01 07:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33621%252F3362002%252F93459621_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33621/3362002/93459621_hd.jpg not downloaded yet
2026-04-01 07:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33621%252F3362002%252F93459612_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33621/3362002/93459612_hd.jpg not downloaded yet
2026-04-01 07:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33621%252F3362002%252F93459620_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33621/3362002/93459620_hd.jpg not downloaded yet
2026-04-01 07:17:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ford-c-max-1-6-vct-titanium-3360417/details> (referer: None)
2026-04-01 07:17:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item f4f2fd4a-bb2b-5e73-8009-ca19f9d68913 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33450/3344954/92887212_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:17:24 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 16d30295-1ae5-55fa-8361-3e05da52e270 with identifier: 3360417
2026-04-01 07:17:24 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 16d30295-1ae5-55fa-8361-3e05da52e270 with scrape type 1
2026-04-01 07:17:24 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 16d30295-1ae5-55fa-8361-3e05da52e270
2026-04-01 07:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33605%252F3360417%252F93587701_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33605/3360417/93587701_hd.jpg not downloaded yet
2026-04-01 07:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33605%252F3360417%252F93587704_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33605/3360417/93587704_hd.jpg not downloaded yet
2026-04-01 07:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33605%252F3360417%252F93587710_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33605/3360417/93587710_hd.jpg not downloaded yet
2026-04-01 07:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33605%252F3360417%252F93587708_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33605/3360417/93587708_hd.jpg not downloaded yet
2026-04-01 07:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33605%252F3360417%252F93587706_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33605/3360417/93587706_hd.jpg not downloaded yet
2026-04-01 07:17:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item f4f2fd4a-bb2b-5e73-8009-ca19f9d68913 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33450/3344954/92887226_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:17:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=75> (referer: None)
2026-04-01 07:17:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item f4a21d95-8635-5cbe-af2c-f0cca9a73744 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33643/3364223/93587899_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:17:27 [autobid.de] INFO: Page 75/145: found 20 items
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361995%27 HTTP/1.1" 200 None
2026-04-01 07:17:27 [autobid.de] INFO: Scrape type for 3361995: 1
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361994%27 HTTP/1.1" 200 None
2026-04-01 07:17:27 [autobid.de] INFO: Scrape type for 3361994: 1
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361998%27 HTTP/1.1" 200 None
2026-04-01 07:17:27 [autobid.de] INFO: Scrape type for 3361998: 1
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362004%27 HTTP/1.1" 200 None
2026-04-01 07:17:27 [autobid.de] INFO: Scrape type for 3362004: 1
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362001%27 HTTP/1.1" 200 None
2026-04-01 07:17:27 [autobid.de] INFO: Scrape type for 3362001: 1
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365541%27 HTTP/1.1" 200 None
2026-04-01 07:17:27 [autobid.de] INFO: Scrape type for 3365541: 1
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365576%27 HTTP/1.1" 200 None
2026-04-01 07:17:27 [autobid.de] INFO: Scrape type for 3365576: 1
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365577%27 HTTP/1.1" 200 None
2026-04-01 07:17:27 [autobid.de] INFO: Scrape type for 3365577: 1
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365578%27 HTTP/1.1" 200 None
2026-04-01 07:17:27 [autobid.de] INFO: Scrape type for 3365578: 1
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365580%27 HTTP/1.1" 200 None
2026-04-01 07:17:27 [autobid.de] INFO: Scrape type for 3365580: 1
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365581%27 HTTP/1.1" 200 None
2026-04-01 07:17:27 [autobid.de] INFO: Scrape type for 3365581: 1
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365585%27 HTTP/1.1" 200 None
2026-04-01 07:17:27 [autobid.de] INFO: Scrape type for 3365585: 1
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365583%27 HTTP/1.1" 200 None
2026-04-01 07:17:27 [autobid.de] INFO: Scrape type for 3365583: 1
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365582%27 HTTP/1.1" 200 None
2026-04-01 07:17:27 [autobid.de] INFO: Scrape type for 3365582: 1
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365584%27 HTTP/1.1" 200 None
2026-04-01 07:17:27 [autobid.de] INFO: Scrape type for 3365584: 1
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365587%27 HTTP/1.1" 200 None
2026-04-01 07:17:27 [autobid.de] INFO: Scrape type for 3365587: 1
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365588%27 HTTP/1.1" 200 None
2026-04-01 07:17:27 [autobid.de] INFO: Scrape type for 3365588: 1
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365590%27 HTTP/1.1" 200 None
2026-04-01 07:17:27 [autobid.de] INFO: Scrape type for 3365590: 1
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365589%27 HTTP/1.1" 200 None
2026-04-01 07:17:27 [autobid.de] INFO: Scrape type for 3365589: 1
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365592%27 HTTP/1.1" 200 None
2026-04-01 07:17:27 [autobid.de] INFO: Scrape type for 3365592: 1
2026-04-01 07:17:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item f4a21d95-8635-5cbe-af2c-f0cca9a73744 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33643/3364223/93587897_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:17:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item f4a21d95-8635-5cbe-af2c-f0cca9a73744 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33643/3364223/93587895_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:17:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-q5-sport-2-0-tfsi-quattro-s-tronic-s-line-3365592/details> (referer: None)
2026-04-01 07:17:30 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: f1450edc-98b8-50a7-912b-0d636d07424c with identifier: 3365592
2026-04-01 07:17:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f1450edc-98b8-50a7-912b-0d636d07424c with scrape type 1
2026-04-01 07:17:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item f1450edc-98b8-50a7-912b-0d636d07424c
2026-04-01 07:17:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365592%252F93590071_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365592/93590071_hd.jpg not downloaded yet
2026-04-01 07:17:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365592%252F93590065_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365592/93590065_hd.jpg not downloaded yet
2026-04-01 07:17:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365592%252F93590064_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365592/93590064_hd.jpg not downloaded yet
2026-04-01 07:17:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365592%252F93590070_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365592/93590070_hd.jpg not downloaded yet
2026-04-01 07:17:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365592%252F93590069_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365592/93590069_hd.jpg not downloaded yet
2026-04-01 07:17:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=76> (referer: None)
2026-04-01 07:17:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-a7-sportback-50-tfsi-e-quattro-s-tronic-s-line-3365589/details> (referer: None)
2026-04-01 07:17:31 [autobid.de] INFO: Page 76/145: found 20 items
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365594%27 HTTP/1.1" 200 None
2026-04-01 07:17:31 [autobid.de] INFO: Scrape type for 3365594: 1
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365593%27 HTTP/1.1" 200 None
2026-04-01 07:17:31 [autobid.de] INFO: Scrape type for 3365593: 1
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365591%27 HTTP/1.1" 200 None
2026-04-01 07:17:31 [autobid.de] INFO: Scrape type for 3365591: 1
2026-04-01 07:17:31 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 32d548c6-d7e4-52b3-8415-701a50980ad0 with identifier: 3365589
2026-04-01 07:17:31 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 32d548c6-d7e4-52b3-8415-701a50980ad0 with scrape type 1
2026-04-01 07:17:31 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 32d548c6-d7e4-52b3-8415-701a50980ad0
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365589%252F93589936_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365589/93589936_hd.jpg not downloaded yet
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365572%27 HTTP/1.1" 200 None
2026-04-01 07:17:31 [autobid.de] INFO: Scrape type for 3365572: 1
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365589%252F93589927_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365589/93589927_hd.jpg not downloaded yet
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365579%27 HTTP/1.1" 200 None
2026-04-01 07:17:31 [autobid.de] INFO: Scrape type for 3365579: 1
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365589%252F93589931_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365589/93589931_hd.jpg not downloaded yet
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365595%27 HTTP/1.1" 200 None
2026-04-01 07:17:31 [autobid.de] INFO: Scrape type for 3365595: 1
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365589%252F93589952_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365589/93589952_hd.jpg not downloaded yet
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365596%27 HTTP/1.1" 200 None
2026-04-01 07:17:31 [autobid.de] INFO: Scrape type for 3365596: 1
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365589%252F93589943_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365589/93589943_hd.jpg not downloaded yet
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353275%27 HTTP/1.1" 200 None
2026-04-01 07:17:31 [autobid.de] INFO: Scrape type for 3353275: 1
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359672%27 HTTP/1.1" 200 None
2026-04-01 07:17:31 [autobid.de] INFO: Scrape type for 3359672: 1
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362058%27 HTTP/1.1" 200 None
2026-04-01 07:17:31 [autobid.de] INFO: Scrape type for 3362058: 1
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366657%27 HTTP/1.1" 200 None
2026-04-01 07:17:31 [autobid.de] INFO: Scrape type for 3366657: 1
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273274796%27 HTTP/1.1" 200 None
2026-04-01 07:17:31 [autobid.de] INFO: Scrape type for 3274796: 1
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366658%27 HTTP/1.1" 200 None
2026-04-01 07:17:31 [autobid.de] INFO: Scrape type for 3366658: 1
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366660%27 HTTP/1.1" 200 None
2026-04-01 07:17:31 [autobid.de] INFO: Scrape type for 3366660: 1
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366659%27 HTTP/1.1" 200 None
2026-04-01 07:17:31 [autobid.de] INFO: Scrape type for 3366659: 1
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353912%27 HTTP/1.1" 200 None
2026-04-01 07:17:31 [autobid.de] INFO: Scrape type for 3353912: 1
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353276%27 HTTP/1.1" 200 None
2026-04-01 07:17:31 [autobid.de] INFO: Scrape type for 3353276: 1
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343689%27 HTTP/1.1" 200 None
2026-04-01 07:17:31 [autobid.de] INFO: Scrape type for 3343689: 1
2026-04-01 07:17:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273312562%27 HTTP/1.1" 200 None
2026-04-01 07:17:32 [autobid.de] INFO: Scrape type for 3312562: 1
2026-04-01 07:17:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273328565%27 HTTP/1.1" 200 None
2026-04-01 07:17:32 [autobid.de] INFO: Scrape type for 3328565: 1
2026-04-01 07:17:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 0a4a2cd1-2ed6-5a84-83f3-1d4d94af97d5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33561/3356013/93368906_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:17:32 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-a6-avant-50-tdi-qu-tiptr-design-ahk-kamera-gra-3353912/details> (referer: None)
2026-04-01 07:17:32 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: eb0d0115-d829-5e85-aa8f-3292dcb4b5f9 with identifier: 3353912
2026-04-01 07:17:32 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item eb0d0115-d829-5e85-aa8f-3292dcb4b5f9 with scrape type 1
2026-04-01 07:17:32 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 3 photos for item eb0d0115-d829-5e85-aa8f-3292dcb4b5f9
2026-04-01 07:17:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33540%252F3353912%252F93180510_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33540/3353912/93180510_hd.jpg not downloaded yet
2026-04-01 07:17:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33540%252F3353912%252F93180511_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33540/3353912/93180511_hd.jpg not downloaded yet
2026-04-01 07:17:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33540%252F3353912%252F93180501_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:17:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33540/3353912/93180501_hd.jpg not downloaded yet
2026-04-01 07:17:35 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=77> (referer: None)
2026-04-01 07:17:35 [autobid.de] INFO: Page 77/145: found 20 items
2026-04-01 07:17:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273349474%27 HTTP/1.1" 200 None
2026-04-01 07:17:35 [autobid.de] INFO: Scrape type for 3349474: 1
2026-04-01 07:17:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273342528%27 HTTP/1.1" 200 None
2026-04-01 07:17:35 [autobid.de] INFO: Scrape type for 3342528: 1
2026-04-01 07:17:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360017%27 HTTP/1.1" 200 None
2026-04-01 07:17:35 [autobid.de] INFO: Scrape type for 3360017: 1
2026-04-01 07:17:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360018%27 HTTP/1.1" 200 None
2026-04-01 07:17:35 [autobid.de] INFO: Scrape type for 3360018: 1
2026-04-01 07:17:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360019%27 HTTP/1.1" 200 None
2026-04-01 07:17:35 [autobid.de] INFO: Scrape type for 3360019: 1
2026-04-01 07:17:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360020%27 HTTP/1.1" 200 None
2026-04-01 07:17:35 [autobid.de] INFO: Scrape type for 3360020: 1
2026-04-01 07:17:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360021%27 HTTP/1.1" 200 None
2026-04-01 07:17:35 [autobid.de] INFO: Scrape type for 3360021: 1
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360022%27 HTTP/1.1" 200 None
2026-04-01 07:17:36 [autobid.de] INFO: Scrape type for 3360022: 1
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359388%27 HTTP/1.1" 200 None
2026-04-01 07:17:36 [autobid.de] INFO: Scrape type for 3359388: 1
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360023%27 HTTP/1.1" 200 None
2026-04-01 07:17:36 [autobid.de] INFO: Scrape type for 3360023: 1
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360024%27 HTTP/1.1" 200 None
2026-04-01 07:17:36 [autobid.de] INFO: Scrape type for 3360024: 1
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360025%27 HTTP/1.1" 200 None
2026-04-01 07:17:36 [autobid.de] INFO: Scrape type for 3360025: 1
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360026%27 HTTP/1.1" 200 None
2026-04-01 07:17:36 [autobid.de] INFO: Scrape type for 3360026: 1
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360027%27 HTTP/1.1" 200 None
2026-04-01 07:17:36 [autobid.de] INFO: Scrape type for 3360027: 1
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360028%27 HTTP/1.1" 200 None
2026-04-01 07:17:36 [autobid.de] INFO: Scrape type for 3360028: 1
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360029%27 HTTP/1.1" 200 None
2026-04-01 07:17:36 [autobid.de] INFO: Scrape type for 3360029: 1
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360030%27 HTTP/1.1" 200 None
2026-04-01 07:17:36 [autobid.de] INFO: Scrape type for 3360030: 1
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360031%27 HTTP/1.1" 200 None
2026-04-01 07:17:36 [autobid.de] INFO: Scrape type for 3360031: 1
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360032%27 HTTP/1.1" 200 None
2026-04-01 07:17:36 [autobid.de] INFO: Scrape type for 3360032: 1
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:17:36 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360033%27 HTTP/1.1" 200 None
2026-04-01 07:17:36 [autobid.de] INFO: Scrape type for 3360033: 1
2026-04-01 07:17:42 [scrapy.extensions.logstats] INFO: Crawled 257 pages (at 10 pages/min), scraped 13 items (at 0 items/min)
2026-04-01 07:17:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 256MiB
2026-04-01 07:18:42 [scrapy.extensions.logstats] INFO: Crawled 257 pages (at 0 pages/min), scraped 13 items (at 0 items/min)
2026-04-01 07:18:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 256MiB
2026-04-01 07:19:42 [scrapy.extensions.logstats] INFO: Crawled 257 pages (at 0 pages/min), scraped 13 items (at 0 items/min)
2026-04-01 07:19:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 256MiB
2026-04-01 07:20:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item d7cab2d4-b8e3-5113-9d8b-707e3a8ddde5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33569/3356886/93376985_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:20:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 1f581311-3a1c-54ed-801a-d65d6dec32b7 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33582/3358181/93376829_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:20:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-a6-allroad-quattro-40-tdi-basis-s-tronic-3360033/details> (referer: None)
2026-04-01 07:20:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 8a70ab58-abbc-5fce-b963-f1f306c4c81f with identifier: 3360033
2026-04-01 07:20:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 8a70ab58-abbc-5fce-b963-f1f306c4c81f with scrape type 1
2026-04-01 07:20:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 8a70ab58-abbc-5fce-b963-f1f306c4c81f
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360033%252F93602503_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360033/93602503_hd.jpg not downloaded yet
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360033%252F93602504_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360033/93602504_hd.jpg not downloaded yet
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360033%252F93602505_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360033/93602505_hd.jpg not downloaded yet
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360033%252F93602506_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360033/93602506_hd.jpg not downloaded yet
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360033%252F93602507_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360033/93602507_hd.jpg not downloaded yet
2026-04-01 07:20:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=78> (referer: None)
2026-04-01 07:20:22 [autobid.de] INFO: Page 78/144: found 20 items
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360037%27 HTTP/1.1" 200 None
2026-04-01 07:20:22 [autobid.de] INFO: Scrape type for 3360037: 1
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360038%27 HTTP/1.1" 200 None
2026-04-01 07:20:22 [autobid.de] INFO: Scrape type for 3360038: 1
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360039%27 HTTP/1.1" 200 None
2026-04-01 07:20:22 [autobid.de] INFO: Scrape type for 3360039: 1
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360040%27 HTTP/1.1" 200 None
2026-04-01 07:20:22 [autobid.de] INFO: Scrape type for 3360040: 1
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362394%27 HTTP/1.1" 200 None
2026-04-01 07:20:22 [autobid.de] INFO: Scrape type for 3362394: 1
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355643%27 HTTP/1.1" 200 None
2026-04-01 07:20:22 [autobid.de] INFO: Scrape type for 3355643: 1
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360530%27 HTTP/1.1" 200 None
2026-04-01 07:20:22 [autobid.de] INFO: Scrape type for 3360530: 1
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362395%27 HTTP/1.1" 200 None
2026-04-01 07:20:22 [autobid.de] INFO: Scrape type for 3362395: 1
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355629%27 HTTP/1.1" 200 None
2026-04-01 07:20:22 [autobid.de] INFO: Scrape type for 3355629: 1
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355633%27 HTTP/1.1" 200 None
2026-04-01 07:20:22 [autobid.de] INFO: Scrape type for 3355633: 1
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362397%27 HTTP/1.1" 200 None
2026-04-01 07:20:22 [autobid.de] INFO: Scrape type for 3362397: 1
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355674%27 HTTP/1.1" 200 None
2026-04-01 07:20:22 [autobid.de] INFO: Scrape type for 3355674: 1
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360528%27 HTTP/1.1" 200 None
2026-04-01 07:20:22 [autobid.de] INFO: Scrape type for 3360528: 1
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362398%27 HTTP/1.1" 200 None
2026-04-01 07:20:22 [autobid.de] INFO: Scrape type for 3362398: 1
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355635%27 HTTP/1.1" 200 None
2026-04-01 07:20:22 [autobid.de] INFO: Scrape type for 3355635: 1
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362556%27 HTTP/1.1" 200 None
2026-04-01 07:20:22 [autobid.de] INFO: Scrape type for 3362556: 1
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355668%27 HTTP/1.1" 200 None
2026-04-01 07:20:22 [autobid.de] INFO: Scrape type for 3355668: 1
2026-04-01 07:20:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361970%27 HTTP/1.1" 200 None
2026-04-01 07:20:23 [autobid.de] INFO: Scrape type for 3361970: 1
2026-04-01 07:20:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362557%27 HTTP/1.1" 200 None
2026-04-01 07:20:23 [autobid.de] INFO: Scrape type for 3362557: 1
2026-04-01 07:20:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355652%27 HTTP/1.1" 200 None
2026-04-01 07:20:23 [autobid.de] INFO: Scrape type for 3355652: 1
2026-04-01 07:20:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/audi-q5-50-tfsi-e-quattro-s-tronic-s-line-3360032/details> (referer: None)
2026-04-01 07:20:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 1091a29c-d5c4-5a69-a11c-4d461a3e4208 with identifier: 3360032
2026-04-01 07:20:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 1091a29c-d5c4-5a69-a11c-4d461a3e4208 with scrape type 1
2026-04-01 07:20:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 1091a29c-d5c4-5a69-a11c-4d461a3e4208
2026-04-01 07:20:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360032%252F93602498_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360032/93602498_hd.jpg not downloaded yet
2026-04-01 07:20:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360032%252F93602499_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360032/93602499_hd.jpg not downloaded yet
2026-04-01 07:20:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360032%252F93602500_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360032/93602500_hd.jpg not downloaded yet
2026-04-01 07:20:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360032%252F93602501_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360032/93602501_hd.jpg not downloaded yet
2026-04-01 07:20:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33601%252F3360032%252F93602502_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33601/3360032/93602502_hd.jpg not downloaded yet
2026-04-01 07:20:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 1f581311-3a1c-54ed-801a-d65d6dec32b7 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33582/3358181/93376792_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:20:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 1f581311-3a1c-54ed-801a-d65d6dec32b7 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33582/3358181/93376802_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:20:25 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=79> (referer: None)
2026-04-01 07:20:25 [autobid.de] INFO: Page 79/144: found 20 items
2026-04-01 07:20:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360531%27 HTTP/1.1" 200 None
2026-04-01 07:20:25 [autobid.de] INFO: Scrape type for 3360531: 1
2026-04-01 07:20:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362558%27 HTTP/1.1" 200 None
2026-04-01 07:20:25 [autobid.de] INFO: Scrape type for 3362558: 1
2026-04-01 07:20:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355656%27 HTTP/1.1" 200 None
2026-04-01 07:20:25 [autobid.de] INFO: Scrape type for 3355656: 1
2026-04-01 07:20:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362559%27 HTTP/1.1" 200 None
2026-04-01 07:20:25 [autobid.de] INFO: Scrape type for 3362559: 1
2026-04-01 07:20:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355678%27 HTTP/1.1" 200 None
2026-04-01 07:20:25 [autobid.de] INFO: Scrape type for 3355678: 1
2026-04-01 07:20:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361971%27 HTTP/1.1" 200 None
2026-04-01 07:20:25 [autobid.de] INFO: Scrape type for 3361971: 1
2026-04-01 07:20:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362560%27 HTTP/1.1" 200 None
2026-04-01 07:20:26 [autobid.de] INFO: Scrape type for 3362560: 1
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360529%27 HTTP/1.1" 200 None
2026-04-01 07:20:26 [autobid.de] INFO: Scrape type for 3360529: 1
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362561%27 HTTP/1.1" 200 None
2026-04-01 07:20:26 [autobid.de] INFO: Scrape type for 3362561: 1
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355481%27 HTTP/1.1" 200 None
2026-04-01 07:20:26 [autobid.de] INFO: Scrape type for 3355481: 1
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362562%27 HTTP/1.1" 200 None
2026-04-01 07:20:26 [autobid.de] INFO: Scrape type for 3362562: 1
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355664%27 HTTP/1.1" 200 None
2026-04-01 07:20:26 [autobid.de] INFO: Scrape type for 3355664: 1
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362032%27 HTTP/1.1" 200 None
2026-04-01 07:20:26 [autobid.de] INFO: Scrape type for 3362032: 1
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354406%27 HTTP/1.1" 200 None
2026-04-01 07:20:26 [autobid.de] INFO: Scrape type for 3354406: 1
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355660%27 HTTP/1.1" 200 None
2026-04-01 07:20:26 [autobid.de] INFO: Scrape type for 3355660: 1
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362564%27 HTTP/1.1" 200 None
2026-04-01 07:20:26 [autobid.de] INFO: Scrape type for 3362564: 1
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355641%27 HTTP/1.1" 200 None
2026-04-01 07:20:26 [autobid.de] INFO: Scrape type for 3355641: 1
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362031%27 HTTP/1.1" 200 None
2026-04-01 07:20:26 [autobid.de] INFO: Scrape type for 3362031: 1
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362399%27 HTTP/1.1" 200 None
2026-04-01 07:20:26 [autobid.de] INFO: Scrape type for 3362399: 1
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355672%27 HTTP/1.1" 200 None
2026-04-01 07:20:26 [autobid.de] INFO: Scrape type for 3355672: 1
2026-04-01 07:20:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 1f581311-3a1c-54ed-801a-d65d6dec32b7 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33582/3358181/93376821_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:20:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item edd97ecd-534d-5a65-ad1a-b284140d0e18 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33444/3344344/93095342_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:20:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=80> (referer: None)
2026-04-01 07:20:29 [autobid.de] INFO: Page 80/144: found 20 items
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362400%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [autobid.de] INFO: Scrape type for 3362400: 1
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355478%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [autobid.de] INFO: Scrape type for 3355478: 1
2026-04-01 07:20:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-polo-1-0-tsi-life-3355672/details> (referer: None)
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362030%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [autobid.de] INFO: Scrape type for 3362030: 1
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362566%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [autobid.de] INFO: Scrape type for 3362566: 1
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355484%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [autobid.de] INFO: Scrape type for 3355484: 1
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362567%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [autobid.de] INFO: Scrape type for 3362567: 1
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355658%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [autobid.de] INFO: Scrape type for 3355658: 1
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362034%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [autobid.de] INFO: Scrape type for 3362034: 1
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362568%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [autobid.de] INFO: Scrape type for 3362568: 1
2026-04-01 07:20:29 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 9bc7e214-d1ac-5017-8872-10f4c4eb97d0 with identifier: 3355672
2026-04-01 07:20:29 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9bc7e214-d1ac-5017-8872-10f4c4eb97d0 with scrape type 1
2026-04-01 07:20:29 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 9bc7e214-d1ac-5017-8872-10f4c4eb97d0
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33557%252F3355672%252F93343541_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33557/3355672/93343541_hd.jpg not downloaded yet
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355676%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [autobid.de] INFO: Scrape type for 3355676: 1
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33557%252F3355672%252F93343545_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33557/3355672/93343545_hd.jpg not downloaded yet
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362401%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [autobid.de] INFO: Scrape type for 3362401: 1
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33557%252F3355672%252F93343559_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33557/3355672/93343559_hd.jpg not downloaded yet
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355681%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [autobid.de] INFO: Scrape type for 3355681: 1
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33557%252F3355672%252F93343555_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33557/3355672/93343555_hd.jpg not downloaded yet
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362029%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [autobid.de] INFO: Scrape type for 3362029: 1
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33557%252F3355672%252F93343550_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33557/3355672/93343550_hd.jpg not downloaded yet
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362402%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [autobid.de] INFO: Scrape type for 3362402: 1
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355482%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [autobid.de] INFO: Scrape type for 3355482: 1
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362403%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [autobid.de] INFO: Scrape type for 3362403: 1
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355662%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [autobid.de] INFO: Scrape type for 3355662: 1
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362035%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [autobid.de] INFO: Scrape type for 3362035: 1
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362404%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [autobid.de] INFO: Scrape type for 3362404: 1
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355647%27 HTTP/1.1" 200 None
2026-04-01 07:20:29 [autobid.de] INFO: Scrape type for 3355647: 1
2026-04-01 07:20:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 748d7a6e-0175-5a99-876e-22e7c46a2ad0 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33621/3362008/93459838_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:20:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-t6-multivan-2-0-tdi-family-3362399/details> (referer: None)
2026-04-01 07:20:30 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 624e84d5-9471-5530-8a68-da37b06dc52c with identifier: 3362399
2026-04-01 07:20:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 624e84d5-9471-5530-8a68-da37b06dc52c with scrape type 1
2026-04-01 07:20:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 624e84d5-9471-5530-8a68-da37b06dc52c
2026-04-01 07:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33624%252F3362399%252F93497081_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33624/3362399/93497081_hd.jpg not downloaded yet
2026-04-01 07:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33624%252F3362399%252F93497082_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33624/3362399/93497082_hd.jpg not downloaded yet
2026-04-01 07:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33624%252F3362399%252F93497093_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33624/3362399/93497093_hd.jpg not downloaded yet
2026-04-01 07:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33624%252F3362399%252F93497089_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33624/3362399/93497089_hd.jpg not downloaded yet
2026-04-01 07:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33624%252F3362399%252F93497083_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33624/3362399/93497083_hd.jpg not downloaded yet
2026-04-01 07:20:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-crafter-35-hochraum-kasten-mr-2-0-tdi-dsg-3362400/details> (referer: None)
2026-04-01 07:20:31 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 4627cee1-7419-5de7-8f53-55a3616b77e3 with identifier: 3362400
2026-04-01 07:20:31 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 4627cee1-7419-5de7-8f53-55a3616b77e3 with scrape type 1
2026-04-01 07:20:31 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 4627cee1-7419-5de7-8f53-55a3616b77e3
2026-04-01 07:20:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33625%252F3362400%252F93496962_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33625/3362400/93496962_hd.jpg not downloaded yet
2026-04-01 07:20:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33625%252F3362400%252F93496963_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33625/3362400/93496963_hd.jpg not downloaded yet
2026-04-01 07:20:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33625%252F3362400%252F93496980_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33625/3362400/93496980_hd.jpg not downloaded yet
2026-04-01 07:20:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33625%252F3362400%252F93496965_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33625/3362400/93496965_hd.jpg not downloaded yet
2026-04-01 07:20:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33625%252F3362400%252F93496964_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33625/3362400/93496964_hd.jpg not downloaded yet
2026-04-01 07:20:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 748d7a6e-0175-5a99-876e-22e7c46a2ad0 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33621/3362008/93459839_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:20:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-golf-variant-1-0-tsi-life-3355647/details> (referer: None)
2026-04-01 07:20:33 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 60c4a6cc-ee39-55dc-afa1-0ad761a2bac7 with identifier: 3355647
2026-04-01 07:20:33 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 60c4a6cc-ee39-55dc-afa1-0ad761a2bac7 with scrape type 1
2026-04-01 07:20:33 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 60c4a6cc-ee39-55dc-afa1-0ad761a2bac7
2026-04-01 07:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33557%252F3355647%252F93372130_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33557/3355647/93372130_hd.jpg not downloaded yet
2026-04-01 07:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33557%252F3355647%252F93372131_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33557/3355647/93372131_hd.jpg not downloaded yet
2026-04-01 07:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33557%252F3355647%252F93372132_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33557/3355647/93372132_hd.jpg not downloaded yet
2026-04-01 07:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33557%252F3355647%252F93372133_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33557/3355647/93372133_hd.jpg not downloaded yet
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33557%252F3355647%252F93372134_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:20:34 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33557/3355647/93372134_hd.jpg not downloaded yet
2026-04-01 07:20:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=81> (referer: None)
2026-04-01 07:20:34 [autobid.de] INFO: Page 81/144: found 20 items
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362405%27 HTTP/1.1" 200 None
2026-04-01 07:20:34 [autobid.de] INFO: Scrape type for 3362405: 1
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355650%27 HTTP/1.1" 200 None
2026-04-01 07:20:34 [autobid.de] INFO: Scrape type for 3355650: 1
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362033%27 HTTP/1.1" 200 None
2026-04-01 07:20:34 [autobid.de] INFO: Scrape type for 3362033: 1
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362565%27 HTTP/1.1" 200 None
2026-04-01 07:20:34 [autobid.de] INFO: Scrape type for 3362565: 1
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355670%27 HTTP/1.1" 200 None
2026-04-01 07:20:34 [autobid.de] INFO: Scrape type for 3355670: 1
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362406%27 HTTP/1.1" 200 None
2026-04-01 07:20:34 [autobid.de] INFO: Scrape type for 3362406: 1
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355475%27 HTTP/1.1" 200 None
2026-04-01 07:20:34 [autobid.de] INFO: Scrape type for 3355475: 1
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361897%27 HTTP/1.1" 200 None
2026-04-01 07:20:34 [autobid.de] INFO: Scrape type for 3361897: 1
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362407%27 HTTP/1.1" 200 None
2026-04-01 07:20:34 [autobid.de] INFO: Scrape type for 3362407: 1
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355639%27 HTTP/1.1" 200 None
2026-04-01 07:20:34 [autobid.de] INFO: Scrape type for 3355639: 1
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362658%27 HTTP/1.1" 200 None
2026-04-01 07:20:34 [autobid.de] INFO: Scrape type for 3362658: 1
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362408%27 HTTP/1.1" 200 None
2026-04-01 07:20:34 [autobid.de] INFO: Scrape type for 3362408: 1
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355483%27 HTTP/1.1" 200 None
2026-04-01 07:20:34 [autobid.de] INFO: Scrape type for 3355483: 1
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362659%27 HTTP/1.1" 200 None
2026-04-01 07:20:34 [autobid.de] INFO: Scrape type for 3362659: 1
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362409%27 HTTP/1.1" 200 None
2026-04-01 07:20:34 [autobid.de] INFO: Scrape type for 3362409: 1
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355479%27 HTTP/1.1" 200 None
2026-04-01 07:20:34 [autobid.de] INFO: Scrape type for 3355479: 1
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362660%27 HTTP/1.1" 200 None
2026-04-01 07:20:34 [autobid.de] INFO: Scrape type for 3362660: 1
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362410%27 HTTP/1.1" 200 None
2026-04-01 07:20:34 [autobid.de] INFO: Scrape type for 3362410: 1
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273355683%27 HTTP/1.1" 200 None
2026-04-01 07:20:34 [autobid.de] INFO: Scrape type for 3355683: 1
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:20:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362661%27 HTTP/1.1" 200 None
2026-04-01 07:20:34 [autobid.de] INFO: Scrape type for 3362661: 1
2026-04-01 07:20:42 [scrapy.extensions.logstats] INFO: Crawled 267 pages (at 10 pages/min), scraped 13 items (at 0 items/min)
2026-04-01 07:20:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 257MiB
2026-04-01 07:21:42 [scrapy.extensions.logstats] INFO: Crawled 267 pages (at 0 pages/min), scraped 13 items (at 0 items/min)
2026-04-01 07:21:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 257MiB
2026-04-01 07:22:42 [scrapy.extensions.logstats] INFO: Crawled 267 pages (at 0 pages/min), scraped 13 items (at 0 items/min)
2026-04-01 07:22:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 257MiB
2026-04-01 07:23:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item e927ffea-0ffc-59c1-9ddc-84698edea946 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33621/3362002/93459620_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:23:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item e927ffea-0ffc-59c1-9ddc-84698edea946 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33621/3362002/93459612_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:23:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mercedes-benz-ml-350-automatik-3362661/details> (referer: None)
2026-04-01 07:23:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 6fdfb044-c975-5af4-8708-6387a51024c8 with identifier: 3362661
2026-04-01 07:23:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6fdfb044-c975-5af4-8708-6387a51024c8 with scrape type 1
2026-04-01 07:23:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 6fdfb044-c975-5af4-8708-6387a51024c8
2026-04-01 07:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362661%252F93480601_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362661/93480601_hd.jpg not downloaded yet
2026-04-01 07:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362661%252F93480594_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362661/93480594_hd.jpg not downloaded yet
2026-04-01 07:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362661%252F93480597_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362661/93480597_hd.jpg not downloaded yet
2026-04-01 07:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362661%252F93480613_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362661/93480613_hd.jpg not downloaded yet
2026-04-01 07:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33627%252F3362661%252F93480596_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33627/3362661/93480596_hd.jpg not downloaded yet
2026-04-01 07:23:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=82> (referer: None)
2026-04-01 07:23:22 [autobid.de] INFO: Page 82/144: found 20 items
2026-04-01 07:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362412%27 HTTP/1.1" 200 None
2026-04-01 07:23:22 [autobid.de] INFO: Scrape type for 3362412: 1
2026-04-01 07:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363520%27 HTTP/1.1" 200 None
2026-04-01 07:23:22 [autobid.de] INFO: Scrape type for 3363520: 1
2026-04-01 07:23:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-tiguan-allspace-2-0-tdi-scr-4motion-dsg-life-3355683/details> (referer: None)
2026-04-01 07:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362413%27 HTTP/1.1" 200 None
2026-04-01 07:23:22 [autobid.de] INFO: Scrape type for 3362413: 1
2026-04-01 07:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363521%27 HTTP/1.1" 200 None
2026-04-01 07:23:22 [autobid.de] INFO: Scrape type for 3363521: 1
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362414%27 HTTP/1.1" 200 None
2026-04-01 07:23:23 [autobid.de] INFO: Scrape type for 3362414: 1
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363518%27 HTTP/1.1" 200 None
2026-04-01 07:23:23 [autobid.de] INFO: Scrape type for 3363518: 1
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362415%27 HTTP/1.1" 200 None
2026-04-01 07:23:23 [autobid.de] INFO: Scrape type for 3362415: 1
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363671%27 HTTP/1.1" 200 None
2026-04-01 07:23:23 [autobid.de] INFO: Scrape type for 3363671: 1
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362416%27 HTTP/1.1" 200 None
2026-04-01 07:23:23 [autobid.de] INFO: Scrape type for 3362416: 1
2026-04-01 07:23:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: b8330ecb-be16-5704-9c88-662f246f7649 with identifier: 3355683
2026-04-01 07:23:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b8330ecb-be16-5704-9c88-662f246f7649 with scrape type 1
2026-04-01 07:23:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item b8330ecb-be16-5704-9c88-662f246f7649
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33557%252F3355683%252F93348867_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33557/3355683/93348867_hd.jpg not downloaded yet
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363674%27 HTTP/1.1" 200 None
2026-04-01 07:23:23 [autobid.de] INFO: Scrape type for 3363674: 1
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33557%252F3355683%252F93348868_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33557/3355683/93348868_hd.jpg not downloaded yet
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362417%27 HTTP/1.1" 200 None
2026-04-01 07:23:23 [autobid.de] INFO: Scrape type for 3362417: 1
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33557%252F3355683%252F93348871_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33557/3355683/93348871_hd.jpg not downloaded yet
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363673%27 HTTP/1.1" 200 None
2026-04-01 07:23:23 [autobid.de] INFO: Scrape type for 3363673: 1
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33557%252F3355683%252F93348870_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33557/3355683/93348870_hd.jpg not downloaded yet
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362418%27 HTTP/1.1" 200 None
2026-04-01 07:23:23 [autobid.de] INFO: Scrape type for 3362418: 1
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33557%252F3355683%252F93348869_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33557/3355683/93348869_hd.jpg not downloaded yet
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363672%27 HTTP/1.1" 200 None
2026-04-01 07:23:23 [autobid.de] INFO: Scrape type for 3363672: 1
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362569%27 HTTP/1.1" 200 None
2026-04-01 07:23:23 [autobid.de] INFO: Scrape type for 3362569: 1
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364450%27 HTTP/1.1" 200 None
2026-04-01 07:23:23 [autobid.de] INFO: Scrape type for 3364450: 1
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362572%27 HTTP/1.1" 200 None
2026-04-01 07:23:23 [autobid.de] INFO: Scrape type for 3362572: 1
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364461%27 HTTP/1.1" 200 None
2026-04-01 07:23:23 [autobid.de] INFO: Scrape type for 3364461: 1
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362419%27 HTTP/1.1" 200 None
2026-04-01 07:23:23 [autobid.de] INFO: Scrape type for 3362419: 1
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362420%27 HTTP/1.1" 200 None
2026-04-01 07:23:23 [autobid.de] INFO: Scrape type for 3362420: 1
2026-04-01 07:23:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 16d30295-1ae5-55fa-8361-3e05da52e270 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33605/3360417/93587706_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:23:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/bentley-continental-gt-6-0-automatik-mulliner-3363520/details> (referer: None)
2026-04-01 07:23:24 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: df3a9743-ece5-5895-9087-9fa994a2a62c with identifier: 3363520
2026-04-01 07:23:24 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item df3a9743-ece5-5895-9087-9fa994a2a62c with scrape type 1
2026-04-01 07:23:24 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item df3a9743-ece5-5895-9087-9fa994a2a62c
2026-04-01 07:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33636%252F3363520%252F93507663_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33636/3363520/93507663_hd.jpg not downloaded yet
2026-04-01 07:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33636%252F3363520%252F93507665_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33636/3363520/93507665_hd.jpg not downloaded yet
2026-04-01 07:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33636%252F3363520%252F93507666_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33636/3363520/93507666_hd.jpg not downloaded yet
2026-04-01 07:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33636%252F3363520%252F93507667_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33636/3363520/93507667_hd.jpg not downloaded yet
2026-04-01 07:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33636%252F3363520%252F93507652_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33636/3363520/93507652_hd.jpg not downloaded yet
2026-04-01 07:23:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 16d30295-1ae5-55fa-8361-3e05da52e270 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33605/3360417/93587708_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:23:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ford-kuga-1-5-ecoboost-automatik-titanium-3362420/details> (referer: None)
2026-04-01 07:23:26 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 57030b18-5de5-5101-a9e2-0d456574e11e with identifier: 3362420
2026-04-01 07:23:26 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 57030b18-5de5-5101-a9e2-0d456574e11e with scrape type 1
2026-04-01 07:23:26 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 57030b18-5de5-5101-a9e2-0d456574e11e
2026-04-01 07:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33625%252F3362420%252F93556406_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33625/3362420/93556406_hd.jpg not downloaded yet
2026-04-01 07:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33625%252F3362420%252F93556409_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33625/3362420/93556409_hd.jpg not downloaded yet
2026-04-01 07:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33625%252F3362420%252F93556426_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33625/3362420/93556426_hd.jpg not downloaded yet
2026-04-01 07:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33625%252F3362420%252F93556415_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33625/3362420/93556415_hd.jpg not downloaded yet
2026-04-01 07:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33625%252F3362420%252F93556411_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33625/3362420/93556411_hd.jpg not downloaded yet
2026-04-01 07:23:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=83> (referer: None)
2026-04-01 07:23:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 16d30295-1ae5-55fa-8361-3e05da52e270 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33605/3360417/93587710_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:23:27 [autobid.de] INFO: Page 83/144: found 20 items
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362421%27 HTTP/1.1" 200 None
2026-04-01 07:23:27 [autobid.de] INFO: Scrape type for 3362421: 1
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362570%27 HTTP/1.1" 200 None
2026-04-01 07:23:27 [autobid.de] INFO: Scrape type for 3362570: 1
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362422%27 HTTP/1.1" 200 None
2026-04-01 07:23:27 [autobid.de] INFO: Scrape type for 3362422: 1
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362423%27 HTTP/1.1" 200 None
2026-04-01 07:23:27 [autobid.de] INFO: Scrape type for 3362423: 1
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362424%27 HTTP/1.1" 200 None
2026-04-01 07:23:27 [autobid.de] INFO: Scrape type for 3362424: 1
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362425%27 HTTP/1.1" 200 None
2026-04-01 07:23:27 [autobid.de] INFO: Scrape type for 3362425: 1
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362427%27 HTTP/1.1" 200 None
2026-04-01 07:23:27 [autobid.de] INFO: Scrape type for 3362427: 1
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362428%27 HTTP/1.1" 200 None
2026-04-01 07:23:27 [autobid.de] INFO: Scrape type for 3362428: 1
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362429%27 HTTP/1.1" 200 None
2026-04-01 07:23:27 [autobid.de] INFO: Scrape type for 3362429: 1
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362430%27 HTTP/1.1" 200 None
2026-04-01 07:23:27 [autobid.de] INFO: Scrape type for 3362430: 1
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362571%27 HTTP/1.1" 200 None
2026-04-01 07:23:27 [autobid.de] INFO: Scrape type for 3362571: 1
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362431%27 HTTP/1.1" 200 None
2026-04-01 07:23:27 [autobid.de] INFO: Scrape type for 3362431: 1
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273321926%27 HTTP/1.1" 200 None
2026-04-01 07:23:27 [autobid.de] INFO: Scrape type for 3321926: 1
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354976%27 HTTP/1.1" 200 None
2026-04-01 07:23:27 [autobid.de] INFO: Scrape type for 3354976: 1
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358961%27 HTTP/1.1" 200 None
2026-04-01 07:23:27 [autobid.de] INFO: Scrape type for 3358961: 1
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357698%27 HTTP/1.1" 200 None
2026-04-01 07:23:27 [autobid.de] INFO: Scrape type for 3357698: 1
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366633%27 HTTP/1.1" 200 None
2026-04-01 07:23:27 [autobid.de] INFO: Scrape type for 3366633: 1
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366010%27 HTTP/1.1" 200 None
2026-04-01 07:23:27 [autobid.de] INFO: Scrape type for 3366010: 1
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363668%27 HTTP/1.1" 200 None
2026-04-01 07:23:27 [autobid.de] INFO: Scrape type for 3363668: 1
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358652%27 HTTP/1.1" 200 None
2026-04-01 07:23:27 [autobid.de] INFO: Scrape type for 3358652: 1
2026-04-01 07:23:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item f1450edc-98b8-50a7-912b-0d636d07424c due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33656/3365592/93590069_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:23:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item f1450edc-98b8-50a7-912b-0d636d07424c due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33656/3365592/93590070_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:23:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mercedes-benz-gla-180-7g-dct-urban-3358652/details> (referer: None)
2026-04-01 07:23:30 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: b29e23fa-a0f3-5af6-abc6-e337717a1e7e with identifier: 3358652
2026-04-01 07:23:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b29e23fa-a0f3-5af6-abc6-e337717a1e7e with scrape type 1
2026-04-01 07:23:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item b29e23fa-a0f3-5af6-abc6-e337717a1e7e
2026-04-01 07:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33587%252F3358652%252F93500262_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33587/3358652/93500262_hd.jpg not downloaded yet
2026-04-01 07:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33587%252F3358652%252F93500263_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33587/3358652/93500263_hd.jpg not downloaded yet
2026-04-01 07:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33587%252F3358652%252F93500264_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33587/3358652/93500264_hd.jpg not downloaded yet
2026-04-01 07:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33587%252F3358652%252F93500265_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33587/3358652/93500265_hd.jpg not downloaded yet
2026-04-01 07:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33587%252F3358652%252F93500266_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33587/3358652/93500266_hd.jpg not downloaded yet
2026-04-01 07:23:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=84> (referer: None)
2026-04-01 07:23:30 [autobid.de] INFO: Page 84/144: found 20 items
2026-04-01 07:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356976%27 HTTP/1.1" 200 None
2026-04-01 07:23:31 [autobid.de] INFO: Scrape type for 3356976: 1
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365095%27 HTTP/1.1" 200 None
2026-04-01 07:23:31 [autobid.de] INFO: Scrape type for 3365095: 1
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362249%27 HTTP/1.1" 200 None
2026-04-01 07:23:31 [autobid.de] INFO: Scrape type for 3362249: 1
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273336728%27 HTTP/1.1" 200 None
2026-04-01 07:23:31 [autobid.de] INFO: Scrape type for 3336728: 1
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362208%27 HTTP/1.1" 200 None
2026-04-01 07:23:31 [autobid.de] INFO: Scrape type for 3362208: 1
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365099%27 HTTP/1.1" 200 None
2026-04-01 07:23:31 [autobid.de] INFO: Scrape type for 3365099: 1
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365098%27 HTTP/1.1" 200 None
2026-04-01 07:23:31 [autobid.de] INFO: Scrape type for 3365098: 1
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273344353%27 HTTP/1.1" 200 None
2026-04-01 07:23:31 [autobid.de] INFO: Scrape type for 3344353: 1
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365096%27 HTTP/1.1" 200 None
2026-04-01 07:23:31 [autobid.de] INFO: Scrape type for 3365096: 1
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273360057%27 HTTP/1.1" 200 None
2026-04-01 07:23:31 [autobid.de] INFO: Scrape type for 3360057: 1
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362244%27 HTTP/1.1" 200 None
2026-04-01 07:23:31 [autobid.de] INFO: Scrape type for 3362244: 1
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358757%27 HTTP/1.1" 200 None
2026-04-01 07:23:31 [autobid.de] INFO: Scrape type for 3358757: 1
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365094%27 HTTP/1.1" 200 None
2026-04-01 07:23:31 [autobid.de] INFO: Scrape type for 3365094: 1
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273344350%27 HTTP/1.1" 200 None
2026-04-01 07:23:31 [autobid.de] INFO: Scrape type for 3344350: 1
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354675%27 HTTP/1.1" 200 None
2026-04-01 07:23:31 [autobid.de] INFO: Scrape type for 3354675: 1
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357011%27 HTTP/1.1" 200 None
2026-04-01 07:23:31 [autobid.de] INFO: Scrape type for 3357011: 1
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356133%27 HTTP/1.1" 200 None
2026-04-01 07:23:31 [autobid.de] INFO: Scrape type for 3356133: 1
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354053%27 HTTP/1.1" 200 None
2026-04-01 07:23:31 [autobid.de] INFO: Scrape type for 3354053: 1
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358650%27 HTTP/1.1" 200 None
2026-04-01 07:23:31 [autobid.de] INFO: Scrape type for 3358650: 1
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356132%27 HTTP/1.1" 200 None
2026-04-01 07:23:31 [autobid.de] INFO: Scrape type for 3356132: 1
2026-04-01 07:23:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mercedes-benz-glc-250-4matic-9g-tronic-3363668/details> (referer: None)
2026-04-01 07:23:31 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 4ea22f46-d017-5a89-a6b2-870993f615d5 with identifier: 3363668
2026-04-01 07:23:32 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 4ea22f46-d017-5a89-a6b2-870993f615d5 with scrape type 1
2026-04-01 07:23:32 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 4ea22f46-d017-5a89-a6b2-870993f615d5
2026-04-01 07:23:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33637%252F3363668%252F93648783_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33637/3363668/93648783_hd.jpg not downloaded yet
2026-04-01 07:23:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33637%252F3363668%252F93648784_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33637/3363668/93648784_hd.jpg not downloaded yet
2026-04-01 07:23:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33637%252F3363668%252F93648785_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33637/3363668/93648785_hd.jpg not downloaded yet
2026-04-01 07:23:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33637%252F3363668%252F93648786_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33637/3363668/93648786_hd.jpg not downloaded yet
2026-04-01 07:23:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33637%252F3363668%252F93648787_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:23:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33637/3363668/93648787_hd.jpg not downloaded yet
2026-04-01 07:23:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item eb0d0115-d829-5e85-aa8f-3292dcb4b5f9 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33540/3353912/93180501_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:23:35 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=85> (referer: None)
2026-04-01 07:23:35 [autobid.de] INFO: Page 85/144: found 20 items
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358960%27 HTTP/1.1" 200 None
2026-04-01 07:23:35 [autobid.de] INFO: Scrape type for 3358960: 1
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357695%27 HTTP/1.1" 200 None
2026-04-01 07:23:35 [autobid.de] INFO: Scrape type for 3357695: 1
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362867%27 HTTP/1.1" 200 None
2026-04-01 07:23:35 [autobid.de] INFO: Scrape type for 3362867: 1
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361946%27 HTTP/1.1" 200 None
2026-04-01 07:23:35 [autobid.de] INFO: Scrape type for 3361946: 1
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362246%27 HTTP/1.1" 200 None
2026-04-01 07:23:35 [autobid.de] INFO: Scrape type for 3362246: 1
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365166%27 HTTP/1.1" 200 None
2026-04-01 07:23:35 [autobid.de] INFO: Scrape type for 3365166: 1
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365564%27 HTTP/1.1" 200 None
2026-04-01 07:23:35 [autobid.de] INFO: Scrape type for 3365564: 1
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363669%27 HTTP/1.1" 200 None
2026-04-01 07:23:35 [autobid.de] INFO: Scrape type for 3363669: 1
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365097%27 HTTP/1.1" 200 None
2026-04-01 07:23:35 [autobid.de] INFO: Scrape type for 3365097: 1
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273319838%27 HTTP/1.1" 200 None
2026-04-01 07:23:35 [autobid.de] INFO: Scrape type for 3319838: 1
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362214%27 HTTP/1.1" 200 None
2026-04-01 07:23:35 [autobid.de] INFO: Scrape type for 3362214: 1
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273350372%27 HTTP/1.1" 200 None
2026-04-01 07:23:35 [autobid.de] INFO: Scrape type for 3350372: 1
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357694%27 HTTP/1.1" 200 None
2026-04-01 07:23:35 [autobid.de] INFO: Scrape type for 3357694: 1
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273359694%27 HTTP/1.1" 200 None
2026-04-01 07:23:35 [autobid.de] INFO: Scrape type for 3359694: 1
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365093%27 HTTP/1.1" 200 None
2026-04-01 07:23:35 [autobid.de] INFO: Scrape type for 3365093: 1
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363995%27 HTTP/1.1" 200 None
2026-04-01 07:23:35 [autobid.de] INFO: Scrape type for 3363995: 1
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362864%27 HTTP/1.1" 200 None
2026-04-01 07:23:35 [autobid.de] INFO: Scrape type for 3362864: 1
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343408%27 HTTP/1.1" 200 None
2026-04-01 07:23:35 [autobid.de] INFO: Scrape type for 3343408: 1
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354676%27 HTTP/1.1" 200 None
2026-04-01 07:23:35 [autobid.de] INFO: Scrape type for 3354676: 1
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:23:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357696%27 HTTP/1.1" 200 None
2026-04-01 07:23:35 [autobid.de] INFO: Scrape type for 3357696: 1
2026-04-01 07:23:42 [scrapy.extensions.logstats] INFO: Crawled 277 pages (at 10 pages/min), scraped 13 items (at 0 items/min)
2026-04-01 07:23:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 259MiB
2026-04-01 07:24:42 [scrapy.extensions.logstats] INFO: Crawled 277 pages (at 0 pages/min), scraped 13 items (at 0 items/min)
2026-04-01 07:24:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 259MiB
2026-04-01 07:25:42 [scrapy.extensions.logstats] INFO: Crawled 277 pages (at 0 pages/min), scraped 13 items (at 0 items/min)
2026-04-01 07:25:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 259MiB
2026-04-01 07:26:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 8a70ab58-abbc-5fce-b963-f1f306c4c81f due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360033/93602507_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:26:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 1091a29c-d5c4-5a69-a11c-4d461a3e4208 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360032/93602502_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:26:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mercedes-benz-a-160-blueefficiency-3357696/details> (referer: None)
2026-04-01 07:26:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 8449c265-f925-528d-b0e0-3fa40161d698 with identifier: 3357696
2026-04-01 07:26:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 8449c265-f925-528d-b0e0-3fa40161d698 with scrape type 1
2026-04-01 07:26:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 8449c265-f925-528d-b0e0-3fa40161d698
2026-04-01 07:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33577%252F3357696%252F93608218_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33577/3357696/93608218_hd.jpg not downloaded yet
2026-04-01 07:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33577%252F3357696%252F93608222_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33577/3357696/93608222_hd.jpg not downloaded yet
2026-04-01 07:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33577%252F3357696%252F93608233_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33577/3357696/93608233_hd.jpg not downloaded yet
2026-04-01 07:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33577%252F3357696%252F93608229_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33577/3357696/93608229_hd.jpg not downloaded yet
2026-04-01 07:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33577%252F3357696%252F93608224_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33577/3357696/93608224_hd.jpg not downloaded yet
2026-04-01 07:26:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=86> (referer: None)
2026-04-01 07:26:23 [autobid.de] INFO: Page 86/144: found 20 items
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365407%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [autobid.de] INFO: Scrape type for 3365407: 1
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354674%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [autobid.de] INFO: Scrape type for 3354674: 1
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358657%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [autobid.de] INFO: Scrape type for 3358657: 1
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362866%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [autobid.de] INFO: Scrape type for 3362866: 1
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273362245%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [autobid.de] INFO: Scrape type for 3362245: 1
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366011%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [autobid.de] INFO: Scrape type for 3366011: 1
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366634%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [autobid.de] INFO: Scrape type for 3366634: 1
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366635%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [autobid.de] INFO: Scrape type for 3366635: 1
2026-04-01 07:26:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/citroen-c3-1-2-puretech-feel-pack-3354676/details> (referer: None)
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356131%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [autobid.de] INFO: Scrape type for 3356131: 1
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273343407%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [autobid.de] INFO: Scrape type for 3343407: 1
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354677%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [autobid.de] INFO: Scrape type for 3354677: 1
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273344320%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [autobid.de] INFO: Scrape type for 3344320: 1
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358665%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [autobid.de] INFO: Scrape type for 3358665: 1
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365193%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [autobid.de] INFO: Scrape type for 3365193: 1
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273327740%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [autobid.de] INFO: Scrape type for 3327740: 1
2026-04-01 07:26:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 01fe2d1e-e266-5250-a6ef-465603f2f093 with identifier: 3354676
2026-04-01 07:26:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 01fe2d1e-e266-5250-a6ef-465603f2f093 with scrape type 1
2026-04-01 07:26:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 01fe2d1e-e266-5250-a6ef-465603f2f093
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33547%252F3354676%252F93207266_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33547/3354676/93207266_hd.jpg not downloaded yet
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363339%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [autobid.de] INFO: Scrape type for 3363339: 1
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33547%252F3354676%252F93207265_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33547/3354676/93207265_hd.jpg not downloaded yet
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363346%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [autobid.de] INFO: Scrape type for 3363346: 1
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33547%252F3354676%252F93207252_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33547/3354676/93207252_hd.jpg not downloaded yet
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361732%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [autobid.de] INFO: Scrape type for 3361732: 1
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33547%252F3354676%252F93207258_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33547/3354676/93207258_hd.jpg not downloaded yet
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365198%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [autobid.de] INFO: Scrape type for 3365198: 1
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33547%252F3354676%252F93207256_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33547/3354676/93207256_hd.jpg not downloaded yet
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361731%27 HTTP/1.1" 200 None
2026-04-01 07:26:23 [autobid.de] INFO: Scrape type for 3361731: 1
2026-04-01 07:26:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 1091a29c-d5c4-5a69-a11c-4d461a3e4208 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360032/93602501_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:26:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-t-cross-1-5-tsi-dsg-iq-drive-3366635/details> (referer: None)
2026-04-01 07:26:24 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: cbb3eea3-db0c-5aa6-a7f7-b57449df5e9a with identifier: 3366635
2026-04-01 07:26:24 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item cbb3eea3-db0c-5aa6-a7f7-b57449df5e9a with scrape type 1
2026-04-01 07:26:24 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item cbb3eea3-db0c-5aa6-a7f7-b57449df5e9a
2026-04-01 07:26:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33667%252F3366635%252F93639253_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33667/3366635/93639253_hd.jpg not downloaded yet
2026-04-01 07:26:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33667%252F3366635%252F93639254_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33667/3366635/93639254_hd.jpg not downloaded yet
2026-04-01 07:26:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33667%252F3366635%252F93639255_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33667/3366635/93639255_hd.jpg not downloaded yet
2026-04-01 07:26:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33667%252F3366635%252F93639256_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33667/3366635/93639256_hd.jpg not downloaded yet
2026-04-01 07:26:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33667%252F3366635%252F93639257_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33667/3366635/93639257_hd.jpg not downloaded yet
2026-04-01 07:26:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 1091a29c-d5c4-5a69-a11c-4d461a3e4208 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33601/3360032/93602500_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:26:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mercedes-benz-a-160-autotronic-classic-3361731/details> (referer: None)
2026-04-01 07:26:27 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 4458bd93-ddd9-58b0-850a-208bdc222a54 with identifier: 3361731
2026-04-01 07:26:27 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 4458bd93-ddd9-58b0-850a-208bdc222a54 with scrape type 1
2026-04-01 07:26:27 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 4458bd93-ddd9-58b0-850a-208bdc222a54
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33618%252F3361731%252F93573468_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33618/3361731/93573468_hd.jpg not downloaded yet
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33618%252F3361731%252F93573469_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33618/3361731/93573469_hd.jpg not downloaded yet
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33618%252F3361731%252F93573475_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33618/3361731/93573475_hd.jpg not downloaded yet
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33618%252F3361731%252F93573474_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33618/3361731/93573474_hd.jpg not downloaded yet
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33618%252F3361731%252F93573472_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33618/3361731/93573472_hd.jpg not downloaded yet
2026-04-01 07:26:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 624e84d5-9471-5530-8a68-da37b06dc52c due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33624/3362399/93497083_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:26:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=87> (referer: None)
2026-04-01 07:26:27 [autobid.de] INFO: Page 87/144: found 20 items
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273357516%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [autobid.de] INFO: Scrape type for 3357516: 1
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356035%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [autobid.de] INFO: Scrape type for 3356035: 1
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366036%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [autobid.de] INFO: Scrape type for 3366036: 1
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353713%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [autobid.de] INFO: Scrape type for 3353713: 1
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365507%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [autobid.de] INFO: Scrape type for 3365507: 1
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363338%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [autobid.de] INFO: Scrape type for 3363338: 1
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364765%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [autobid.de] INFO: Scrape type for 3364765: 1
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356031%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [autobid.de] INFO: Scrape type for 3356031: 1
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366306%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [autobid.de] INFO: Scrape type for 3366306: 1
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365508%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [autobid.de] INFO: Scrape type for 3365508: 1
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366774%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [autobid.de] INFO: Scrape type for 3366774: 1
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273317675%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [autobid.de] INFO: Scrape type for 3317675: 1
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363353%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [autobid.de] INFO: Scrape type for 3363353: 1
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356029%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [autobid.de] INFO: Scrape type for 3356029: 1
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363492%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [autobid.de] INFO: Scrape type for 3363492: 1
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363491%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [autobid.de] INFO: Scrape type for 3363491: 1
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273294536%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [autobid.de] INFO: Scrape type for 3294536: 1
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363356%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [autobid.de] INFO: Scrape type for 3363356: 1
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363360%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [autobid.de] INFO: Scrape type for 3363360: 1
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363351%27 HTTP/1.1" 200 None
2026-04-01 07:26:27 [autobid.de] INFO: Scrape type for 3363351: 1
2026-04-01 07:26:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 4627cee1-7419-5de7-8f53-55a3616b77e3 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33625/3362400/93496964_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:26:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 4627cee1-7419-5de7-8f53-55a3616b77e3 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33625/3362400/93496965_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:26:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/seat-leon-sportstourer-1-4-tsi-e-dsg-xcellence-3363351/details> (referer: None)
2026-04-01 07:26:30 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 54b3d75b-b470-58f2-b58c-c4ab625b2e42 with identifier: 3363351
2026-04-01 07:26:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 54b3d75b-b470-58f2-b58c-c4ab625b2e42 with scrape type 1
2026-04-01 07:26:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 54b3d75b-b470-58f2-b58c-c4ab625b2e42
2026-04-01 07:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33634%252F3363351%252F93547087_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33634/3363351/93547087_hd.jpg not downloaded yet
2026-04-01 07:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33634%252F3363351%252F93547088_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33634/3363351/93547088_hd.jpg not downloaded yet
2026-04-01 07:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33634%252F3363351%252F93547089_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33634/3363351/93547089_hd.jpg not downloaded yet
2026-04-01 07:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33634%252F3363351%252F93547090_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33634/3363351/93547090_hd.jpg not downloaded yet
2026-04-01 07:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33634%252F3363351%252F93547091_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33634/3363351/93547091_hd.jpg not downloaded yet
2026-04-01 07:26:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=88> (referer: None)
2026-04-01 07:26:30 [autobid.de] INFO: Page 88/144: found 20 items
2026-04-01 07:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361730%27 HTTP/1.1" 200 None
2026-04-01 07:26:30 [autobid.de] INFO: Scrape type for 3361730: 1
2026-04-01 07:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366035%27 HTTP/1.1" 200 None
2026-04-01 07:26:30 [autobid.de] INFO: Scrape type for 3366035: 1
2026-04-01 07:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366953%27 HTTP/1.1" 200 None
2026-04-01 07:26:30 [autobid.de] INFO: Scrape type for 3366953: 1
2026-04-01 07:26:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363354%27 HTTP/1.1" 200 None
2026-04-01 07:26:31 [autobid.de] INFO: Scrape type for 3363354: 1
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356032%27 HTTP/1.1" 200 None
2026-04-01 07:26:31 [autobid.de] INFO: Scrape type for 3356032: 1
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273356030%27 HTTP/1.1" 200 None
2026-04-01 07:26:31 [autobid.de] INFO: Scrape type for 3356030: 1
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365022%27 HTTP/1.1" 200 None
2026-04-01 07:26:31 [autobid.de] INFO: Scrape type for 3365022: 1
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365462%27 HTTP/1.1" 200 None
2026-04-01 07:26:31 [autobid.de] INFO: Scrape type for 3365462: 1
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365456%27 HTTP/1.1" 200 None
2026-04-01 07:26:31 [autobid.de] INFO: Scrape type for 3365456: 1
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366327%27 HTTP/1.1" 200 None
2026-04-01 07:26:31 [autobid.de] INFO: Scrape type for 3366327: 1
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353708%27 HTTP/1.1" 200 None
2026-04-01 07:26:31 [autobid.de] INFO: Scrape type for 3353708: 1
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366952%27 HTTP/1.1" 200 None
2026-04-01 07:26:31 [autobid.de] INFO: Scrape type for 3366952: 1
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366326%27 HTTP/1.1" 200 None
2026-04-01 07:26:31 [autobid.de] INFO: Scrape type for 3366326: 1
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366316%27 HTTP/1.1" 200 None
2026-04-01 07:26:31 [autobid.de] INFO: Scrape type for 3366316: 1
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365021%27 HTTP/1.1" 200 None
2026-04-01 07:26:31 [autobid.de] INFO: Scrape type for 3365021: 1
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363493%27 HTTP/1.1" 200 None
2026-04-01 07:26:31 [autobid.de] INFO: Scrape type for 3363493: 1
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361340%27 HTTP/1.1" 200 None
2026-04-01 07:26:31 [autobid.de] INFO: Scrape type for 3361340: 1
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365761%27 HTTP/1.1" 200 None
2026-04-01 07:26:31 [autobid.de] INFO: Scrape type for 3365761: 1
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365762%27 HTTP/1.1" 200 None
2026-04-01 07:26:31 [autobid.de] INFO: Scrape type for 3365762: 1
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365774%27 HTTP/1.1" 200 None
2026-04-01 07:26:31 [autobid.de] INFO: Scrape type for 3365774: 1
2026-04-01 07:26:32 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/peugeot-508-sw-1-5-blue-hdi-eat8-allure-pack-3363360/details> (referer: None)
2026-04-01 07:26:32 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: f8e8306e-fe29-56a3-aebe-db65a83f44ed with identifier: 3363360
2026-04-01 07:26:32 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item f8e8306e-fe29-56a3-aebe-db65a83f44ed with scrape type 1
2026-04-01 07:26:32 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item f8e8306e-fe29-56a3-aebe-db65a83f44ed
2026-04-01 07:26:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33634%252F3363360%252F93567672_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33634/3363360/93567672_hd.jpg not downloaded yet
2026-04-01 07:26:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33634%252F3363360%252F93567673_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33634/3363360/93567673_hd.jpg not downloaded yet
2026-04-01 07:26:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33634%252F3363360%252F93567674_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33634/3363360/93567674_hd.jpg not downloaded yet
2026-04-01 07:26:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33634%252F3363360%252F93567675_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33634/3363360/93567675_hd.jpg not downloaded yet
2026-04-01 07:26:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33634%252F3363360%252F93567676_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:26:32 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33634/3363360/93567676_hd.jpg not downloaded yet
2026-04-01 07:26:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 60c4a6cc-ee39-55dc-afa1-0ad761a2bac7 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33557/3355647/93372134_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:26:35 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=89> (referer: None)
2026-04-01 07:26:35 [autobid.de] INFO: Page 89/144: found 20 items
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365773%27 HTTP/1.1" 200 None
2026-04-01 07:26:35 [autobid.de] INFO: Scrape type for 3365773: 1
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363344%27 HTTP/1.1" 200 None
2026-04-01 07:26:35 [autobid.de] INFO: Scrape type for 3363344: 1
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361341%27 HTTP/1.1" 200 None
2026-04-01 07:26:35 [autobid.de] INFO: Scrape type for 3361341: 1
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363363%27 HTTP/1.1" 200 None
2026-04-01 07:26:35 [autobid.de] INFO: Scrape type for 3363363: 1
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363364%27 HTTP/1.1" 200 None
2026-04-01 07:26:35 [autobid.de] INFO: Scrape type for 3363364: 1
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363376%27 HTTP/1.1" 200 None
2026-04-01 07:26:35 [autobid.de] INFO: Scrape type for 3363376: 1
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363370%27 HTTP/1.1" 200 None
2026-04-01 07:26:35 [autobid.de] INFO: Scrape type for 3363370: 1
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353707%27 HTTP/1.1" 200 None
2026-04-01 07:26:35 [autobid.de] INFO: Scrape type for 3353707: 1
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363359%27 HTTP/1.1" 200 None
2026-04-01 07:26:35 [autobid.de] INFO: Scrape type for 3363359: 1
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363365%27 HTTP/1.1" 200 None
2026-04-01 07:26:35 [autobid.de] INFO: Scrape type for 3363365: 1
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363341%27 HTTP/1.1" 200 None
2026-04-01 07:26:35 [autobid.de] INFO: Scrape type for 3363341: 1
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365504%27 HTTP/1.1" 200 None
2026-04-01 07:26:35 [autobid.de] INFO: Scrape type for 3365504: 1
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363355%27 HTTP/1.1" 200 None
2026-04-01 07:26:35 [autobid.de] INFO: Scrape type for 3363355: 1
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366311%27 HTTP/1.1" 200 None
2026-04-01 07:26:35 [autobid.de] INFO: Scrape type for 3366311: 1
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365667%27 HTTP/1.1" 200 None
2026-04-01 07:26:35 [autobid.de] INFO: Scrape type for 3365667: 1
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361342%27 HTTP/1.1" 200 None
2026-04-01 07:26:35 [autobid.de] INFO: Scrape type for 3361342: 1
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365500%27 HTTP/1.1" 200 None
2026-04-01 07:26:35 [autobid.de] INFO: Scrape type for 3365500: 1
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363368%27 HTTP/1.1" 200 None
2026-04-01 07:26:35 [autobid.de] INFO: Scrape type for 3363368: 1
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365668%27 HTTP/1.1" 200 None
2026-04-01 07:26:35 [autobid.de] INFO: Scrape type for 3365668: 1
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:26:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366307%27 HTTP/1.1" 200 None
2026-04-01 07:26:35 [autobid.de] INFO: Scrape type for 3366307: 1
2026-04-01 07:26:42 [scrapy.extensions.logstats] INFO: Crawled 287 pages (at 10 pages/min), scraped 13 items (at 0 items/min)
2026-04-01 07:26:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 260MiB
2026-04-01 07:27:42 [scrapy.extensions.logstats] INFO: Crawled 287 pages (at 0 pages/min), scraped 13 items (at 0 items/min)
2026-04-01 07:27:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 260MiB
2026-04-01 07:28:42 [scrapy.extensions.logstats] INFO: Crawled 287 pages (at 0 pages/min), scraped 13 items (at 0 items/min)
2026-04-01 07:28:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 260MiB
2026-04-01 07:29:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 6fdfb044-c975-5af4-8708-6387a51024c8 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33627/3362661/93480596_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:29:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item df3a9743-ece5-5895-9087-9fa994a2a62c due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33636/3363520/93507652_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:29:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/knaus-van-ti-650-meg-first-edition-3366307/details> (referer: None)
2026-04-01 07:29:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 77e8defe-6c06-50a9-a5ab-c256d7ab6edc with identifier: 3366307
2026-04-01 07:29:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 77e8defe-6c06-50a9-a5ab-c256d7ab6edc with scrape type 1
2026-04-01 07:29:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 77e8defe-6c06-50a9-a5ab-c256d7ab6edc
2026-04-01 07:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33664%252F3366307%252F93611051_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:29:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33664/3366307/93611051_hd.jpg not downloaded yet
2026-04-01 07:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33664%252F3366307%252F93611043_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:29:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33664/3366307/93611043_hd.jpg not downloaded yet
2026-04-01 07:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33664%252F3366307%252F93611031_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:29:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33664/3366307/93611031_hd.jpg not downloaded yet
2026-04-01 07:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33664%252F3366307%252F93611034_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:29:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33664/3366307/93611034_hd.jpg not downloaded yet
2026-04-01 07:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33664%252F3366307%252F93611019_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:29:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33664/3366307/93611019_hd.jpg not downloaded yet
2026-04-01 07:29:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=90> (referer: None)
2026-04-01 07:29:22 [autobid.de] INFO: Page 90/144: found 20 items
2026-04-01 07:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363357%27 HTTP/1.1" 200 None
2026-04-01 07:29:22 [autobid.de] INFO: Scrape type for 3363357: 1
2026-04-01 07:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363345%27 HTTP/1.1" 200 None
2026-04-01 07:29:22 [autobid.de] INFO: Scrape type for 3363345: 1
2026-04-01 07:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364758%27 HTTP/1.1" 200 None
2026-04-01 07:29:22 [autobid.de] INFO: Scrape type for 3364758: 1
2026-04-01 07:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273350841%27 HTTP/1.1" 200 None
2026-04-01 07:29:22 [autobid.de] INFO: Scrape type for 3350841: 1
2026-04-01 07:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363369%27 HTTP/1.1" 200 None
2026-04-01 07:29:23 [autobid.de] INFO: Scrape type for 3363369: 1
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363372%27 HTTP/1.1" 200 None
2026-04-01 07:29:23 [autobid.de] INFO: Scrape type for 3363372: 1
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363375%27 HTTP/1.1" 200 None
2026-04-01 07:29:23 [autobid.de] INFO: Scrape type for 3363375: 1
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364709%27 HTTP/1.1" 200 None
2026-04-01 07:29:23 [autobid.de] INFO: Scrape type for 3364709: 1
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364749%27 HTTP/1.1" 200 None
2026-04-01 07:29:23 [autobid.de] INFO: Scrape type for 3364749: 1
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364757%27 HTTP/1.1" 200 None
2026-04-01 07:29:23 [autobid.de] INFO: Scrape type for 3364757: 1
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365769%27 HTTP/1.1" 200 None
2026-04-01 07:29:23 [autobid.de] INFO: Scrape type for 3365769: 1
2026-04-01 07:29:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ferrari-296-gts-3365668/details> (referer: None)
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365775%27 HTTP/1.1" 200 None
2026-04-01 07:29:23 [autobid.de] INFO: Scrape type for 3365775: 1
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365498%27 HTTP/1.1" 200 None
2026-04-01 07:29:23 [autobid.de] INFO: Scrape type for 3365498: 1
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273361399%27 HTTP/1.1" 200 None
2026-04-01 07:29:23 [autobid.de] INFO: Scrape type for 3361399: 1
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365089%27 HTTP/1.1" 200 None
2026-04-01 07:29:23 [autobid.de] INFO: Scrape type for 3365089: 1
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365503%27 HTTP/1.1" 200 None
2026-04-01 07:29:23 [autobid.de] INFO: Scrape type for 3365503: 1
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363358%27 HTTP/1.1" 200 None
2026-04-01 07:29:23 [autobid.de] INFO: Scrape type for 3363358: 1
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365510%27 HTTP/1.1" 200 None
2026-04-01 07:29:23 [autobid.de] INFO: Scrape type for 3365510: 1
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273353705%27 HTTP/1.1" 200 None
2026-04-01 07:29:23 [autobid.de] INFO: Scrape type for 3353705: 1
2026-04-01 07:29:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 9bd0ff8c-f433-5d6f-a458-9ccce5b2c4c7 with identifier: 3365668
2026-04-01 07:29:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 9bd0ff8c-f433-5d6f-a458-9ccce5b2c4c7 with scrape type 1
2026-04-01 07:29:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 9bd0ff8c-f433-5d6f-a458-9ccce5b2c4c7
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33657%252F3365668%252F93599774_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:29:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33657/3365668/93599774_hd.jpg not downloaded yet
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365505%27 HTTP/1.1" 200 None
2026-04-01 07:29:23 [autobid.de] INFO: Scrape type for 3365505: 1
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33657%252F3365668%252F93599766_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:29:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33657/3365668/93599766_hd.jpg not downloaded yet
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33657%252F3365668%252F93599771_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:29:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33657/3365668/93599771_hd.jpg not downloaded yet
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33657%252F3365668%252F93599770_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:29:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33657/3365668/93599770_hd.jpg not downloaded yet
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33657%252F3365668%252F93599778_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:29:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33657/3365668/93599778_hd.jpg not downloaded yet
2026-04-01 07:29:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item df3a9743-ece5-5895-9087-9fa994a2a62c due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33636/3363520/93507667_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:29:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/kia-ev6-gt-4wd-3365769/details> (referer: None)
2026-04-01 07:29:24 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: bc95e64b-7b82-5f67-b253-ad765166b866 with identifier: 3365769
2026-04-01 07:29:24 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item bc95e64b-7b82-5f67-b253-ad765166b866 with scrape type 1
2026-04-01 07:29:24 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item bc95e64b-7b82-5f67-b253-ad765166b866
2026-04-01 07:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33658%252F3365769%252F93608925_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:29:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33658/3365769/93608925_hd.jpg not downloaded yet
2026-04-01 07:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33658%252F3365769%252F93608919_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:29:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33658/3365769/93608919_hd.jpg not downloaded yet
2026-04-01 07:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33658%252F3365769%252F93608936_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:29:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33658/3365769/93608936_hd.jpg not downloaded yet
2026-04-01 07:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33658%252F3365769%252F93608935_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:29:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33658/3365769/93608935_hd.jpg not downloaded yet
2026-04-01 07:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:29:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33658%252F3365769%252F93608933_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:29:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33658/3365769/93608933_hd.jpg not downloaded yet
2026-04-01 07:29:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 57030b18-5de5-5101-a9e2-0d456574e11e due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33625/3362420/93556411_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:29:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 57030b18-5de5-5101-a9e2-0d456574e11e due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33625/3362420/93556415_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:29:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item b29e23fa-a0f3-5af6-abc6-e337717a1e7e due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33587/3358652/93500266_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:29:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 4ea22f46-d017-5a89-a6b2-870993f615d5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33637/3363668/93648783_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:29:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 4ea22f46-d017-5a89-a6b2-870993f615d5 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33637/3363668/93648787_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:29:42 [scrapy.extensions.logstats] INFO: Crawled 291 pages (at 4 pages/min), scraped 13 items (at 0 items/min)
2026-04-01 07:29:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 261MiB
2026-04-01 07:30:42 [scrapy.extensions.logstats] INFO: Crawled 291 pages (at 0 pages/min), scraped 13 items (at 0 items/min)
2026-04-01 07:30:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 261MiB
2026-04-01 07:31:42 [scrapy.extensions.logstats] INFO: Crawled 291 pages (at 0 pages/min), scraped 13 items (at 0 items/min)
2026-04-01 07:31:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 261MiB
2026-04-01 07:32:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 8449c265-f925-528d-b0e0-3fa40161d698 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33577/3357696/93608224_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:32:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item cbb3eea3-db0c-5aa6-a7f7-b57449df5e9a due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33667/3366635/93639257_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:32:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=91> (referer: None)
2026-04-01 07:32:24 [autobid.de] INFO: Page 91/144: found 20 items
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365785%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [autobid.de] INFO: Scrape type for 3365785: 1
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365781%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [autobid.de] INFO: Scrape type for 3365781: 1
2026-04-01 07:32:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item cbb3eea3-db0c-5aa6-a7f7-b57449df5e9a due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33667/3366635/93639256_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363362%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [autobid.de] INFO: Scrape type for 3363362: 1
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273317677%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [autobid.de] INFO: Scrape type for 3317677: 1
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366319%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [autobid.de] INFO: Scrape type for 3366319: 1
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363735%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [autobid.de] INFO: Scrape type for 3363735: 1
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363736%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [autobid.de] INFO: Scrape type for 3363736: 1
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364672%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [autobid.de] INFO: Scrape type for 3364672: 1
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364671%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [autobid.de] INFO: Scrape type for 3364671: 1
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363737%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [autobid.de] INFO: Scrape type for 3363737: 1
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363738%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [autobid.de] INFO: Scrape type for 3363738: 1
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363739%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [autobid.de] INFO: Scrape type for 3363739: 1
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365252%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [autobid.de] INFO: Scrape type for 3365252: 1
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358842%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [autobid.de] INFO: Scrape type for 3358842: 1
2026-04-01 07:32:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/bmw-118i-advantage-3365505/details> (referer: None)
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273336681%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [autobid.de] INFO: Scrape type for 3336681: 1
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365197%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [autobid.de] INFO: Scrape type for 3365197: 1
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365196%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [autobid.de] INFO: Scrape type for 3365196: 1
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363740%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [autobid.de] INFO: Scrape type for 3363740: 1
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363741%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [autobid.de] INFO: Scrape type for 3363741: 1
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363742%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [autobid.de] INFO: Scrape type for 3363742: 1
2026-04-01 07:32:24 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: b037c7a8-46d7-5ee8-92f7-82ca97e2df64 with identifier: 3365505
2026-04-01 07:32:24 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item b037c7a8-46d7-5ee8-92f7-82ca97e2df64 with scrape type 1
2026-04-01 07:32:24 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item b037c7a8-46d7-5ee8-92f7-82ca97e2df64
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365505%252F93640368_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365505/93640368_hd.jpg not downloaded yet
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365505%252F93640369_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365505/93640369_hd.jpg not downloaded yet
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365505%252F93640370_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365505/93640370_hd.jpg not downloaded yet
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365505%252F93640371_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365505/93640371_hd.jpg not downloaded yet
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365505%252F93640372_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:32:24 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365505/93640372_hd.jpg not downloaded yet
2026-04-01 07:32:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 4458bd93-ddd9-58b0-850a-208bdc222a54 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33618/3361731/93573472_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:32:25 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/opel-corsa-1-5-cdti-edition-3358842/details> (referer: None)
2026-04-01 07:32:25 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 3fe92428-0159-576f-99a1-15baefeda4fc with identifier: 3358842
2026-04-01 07:32:25 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 3fe92428-0159-576f-99a1-15baefeda4fc with scrape type 1
2026-04-01 07:32:25 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 3fe92428-0159-576f-99a1-15baefeda4fc
2026-04-01 07:32:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33589%252F3358842%252F93461884_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:32:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33589/3358842/93461884_hd.jpg not downloaded yet
2026-04-01 07:32:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33589%252F3358842%252F93464445_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:32:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33589/3358842/93464445_hd.jpg not downloaded yet
2026-04-01 07:32:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33589%252F3358842%252F93464446_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:32:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33589/3358842/93464446_hd.jpg not downloaded yet
2026-04-01 07:32:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33589%252F3358842%252F93464447_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:32:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33589/3358842/93464447_hd.jpg not downloaded yet
2026-04-01 07:32:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:32:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33589%252F3358842%252F93464448_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:32:26 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33589/3358842/93464448_hd.jpg not downloaded yet
2026-04-01 07:32:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 4458bd93-ddd9-58b0-850a-208bdc222a54 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33618/3361731/93573474_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:32:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 54b3d75b-b470-58f2-b58c-c4ab625b2e42 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33634/3363351/93547091_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:32:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item f8e8306e-fe29-56a3-aebe-db65a83f44ed due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33634/3363360/93567676_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:32:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item f8e8306e-fe29-56a3-aebe-db65a83f44ed due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33634/3363360/93567675_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:32:42 [scrapy.extensions.logstats] INFO: Crawled 294 pages (at 3 pages/min), scraped 13 items (at 0 items/min)
2026-04-01 07:32:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 261MiB
2026-04-01 07:33:42 [scrapy.extensions.logstats] INFO: Crawled 294 pages (at 0 pages/min), scraped 13 items (at 0 items/min)
2026-04-01 07:33:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 261MiB
2026-04-01 07:34:42 [scrapy.extensions.logstats] INFO: Crawled 294 pages (at 0 pages/min), scraped 13 items (at 0 items/min)
2026-04-01 07:34:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 261MiB
2026-04-01 07:35:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 77e8defe-6c06-50a9-a5ab-c256d7ab6edc due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33664/3366307/93611019_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:35:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 9bd0ff8c-f433-5d6f-a458-9ccce5b2c4c7 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33657/3365668/93599778_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:35:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 9bd0ff8c-f433-5d6f-a458-9ccce5b2c4c7 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33657/3365668/93599770_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:35:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item bc95e64b-7b82-5f67-b253-ad765166b866 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33658/3365769/93608933_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:35:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item bc95e64b-7b82-5f67-b253-ad765166b866 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33658/3365769/93608935_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:35:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item bc95e64b-7b82-5f67-b253-ad765166b866 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33658/3365769/93608936_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:35:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-golf-variant-2-0-tdi-life-3363742/details> (referer: None)
2026-04-01 07:35:29 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 1fe1caae-b816-5136-b54b-bd539219d9dd with identifier: 3363742
2026-04-01 07:35:29 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 1fe1caae-b816-5136-b54b-bd539219d9dd with scrape type 1
2026-04-01 07:35:29 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 1fe1caae-b816-5136-b54b-bd539219d9dd
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33638%252F3363742%252F93537343_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:35:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33638/3363742/93537343_hd.jpg not downloaded yet
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33638%252F3363742%252F93537351_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:35:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33638/3363742/93537351_hd.jpg not downloaded yet
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33638%252F3363742%252F93537350_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:35:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33638/3363742/93537350_hd.jpg not downloaded yet
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33638%252F3363742%252F93537357_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:35:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33638/3363742/93537357_hd.jpg not downloaded yet
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33638%252F3363742%252F93537361_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:35:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33638/3363742/93537361_hd.jpg not downloaded yet
2026-04-01 07:35:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=92> (referer: None)
2026-04-01 07:35:29 [autobid.de] INFO: Page 92/144: found 20 items
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364512%27 HTTP/1.1" 200 None
2026-04-01 07:35:29 [autobid.de] INFO: Scrape type for 3364512: 1
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363371%27 HTTP/1.1" 200 None
2026-04-01 07:35:29 [autobid.de] INFO: Scrape type for 3363371: 1
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366037%27 HTTP/1.1" 200 None
2026-04-01 07:35:29 [autobid.de] INFO: Scrape type for 3366037: 1
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363700%27 HTTP/1.1" 200 None
2026-04-01 07:35:29 [autobid.de] INFO: Scrape type for 3363700: 1
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364748%27 HTTP/1.1" 200 None
2026-04-01 07:35:29 [autobid.de] INFO: Scrape type for 3364748: 1
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365160%27 HTTP/1.1" 200 None
2026-04-01 07:35:29 [autobid.de] INFO: Scrape type for 3365160: 1
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365161%27 HTTP/1.1" 200 None
2026-04-01 07:35:29 [autobid.de] INFO: Scrape type for 3365161: 1
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364513%27 HTTP/1.1" 200 None
2026-04-01 07:35:29 [autobid.de] INFO: Scrape type for 3364513: 1
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364514%27 HTTP/1.1" 200 None
2026-04-01 07:35:29 [autobid.de] INFO: Scrape type for 3364514: 1
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364515%27 HTTP/1.1" 200 None
2026-04-01 07:35:29 [autobid.de] INFO: Scrape type for 3364515: 1
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364520%27 HTTP/1.1" 200 None
2026-04-01 07:35:29 [autobid.de] INFO: Scrape type for 3364520: 1
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364658%27 HTTP/1.1" 200 None
2026-04-01 07:35:29 [autobid.de] INFO: Scrape type for 3364658: 1
2026-04-01 07:35:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364660%27 HTTP/1.1" 200 None
2026-04-01 07:35:30 [autobid.de] INFO: Scrape type for 3364660: 1
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364662%27 HTTP/1.1" 200 None
2026-04-01 07:35:30 [autobid.de] INFO: Scrape type for 3364662: 1
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364666%27 HTTP/1.1" 200 None
2026-04-01 07:35:30 [autobid.de] INFO: Scrape type for 3364666: 1
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363699%27 HTTP/1.1" 200 None
2026-04-01 07:35:30 [autobid.de] INFO: Scrape type for 3363699: 1
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365195%27 HTTP/1.1" 200 None
2026-04-01 07:35:30 [autobid.de] INFO: Scrape type for 3365195: 1
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365194%27 HTTP/1.1" 200 None
2026-04-01 07:35:30 [autobid.de] INFO: Scrape type for 3365194: 1
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364517%27 HTTP/1.1" 200 None
2026-04-01 07:35:30 [autobid.de] INFO: Scrape type for 3364517: 1
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364518%27 HTTP/1.1" 200 None
2026-04-01 07:35:30 [autobid.de] INFO: Scrape type for 3364518: 1
2026-04-01 07:35:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item bc95e64b-7b82-5f67-b253-ad765166b866 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33658/3365769/93608919_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:35:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/opel-astra-sports-tourer-1-6-cdti-automatik-business-3363741/details> (referer: None)
2026-04-01 07:35:30 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 50a6746d-8840-5a0b-9dcb-616de15a4322 with identifier: 3363741
2026-04-01 07:35:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 50a6746d-8840-5a0b-9dcb-616de15a4322 with scrape type 1
2026-04-01 07:35:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 50a6746d-8840-5a0b-9dcb-616de15a4322
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33638%252F3363741%252F93537311_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:35:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33638/3363741/93537311_hd.jpg not downloaded yet
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33638%252F3363741%252F93537295_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:35:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33638/3363741/93537295_hd.jpg not downloaded yet
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33638%252F3363741%252F93537293_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:35:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33638/3363741/93537293_hd.jpg not downloaded yet
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33638%252F3363741%252F93537294_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:35:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33638/3363741/93537294_hd.jpg not downloaded yet
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33638%252F3363741%252F93537305_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:35:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33638/3363741/93537305_hd.jpg not downloaded yet
2026-04-01 07:35:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-golf-1-5-tsi-act-life-3364517/details> (referer: None)
2026-04-01 07:35:31 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 39d00a86-abd5-5d7e-ac06-d16494325f00 with identifier: 3364517
2026-04-01 07:35:31 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 39d00a86-abd5-5d7e-ac06-d16494325f00 with scrape type 1
2026-04-01 07:35:31 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 39d00a86-abd5-5d7e-ac06-d16494325f00
2026-04-01 07:35:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364517%252F93537580_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:35:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364517/93537580_hd.jpg not downloaded yet
2026-04-01 07:35:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364517%252F93537575_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:35:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364517/93537575_hd.jpg not downloaded yet
2026-04-01 07:35:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364517%252F93537578_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:35:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364517/93537578_hd.jpg not downloaded yet
2026-04-01 07:35:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364517%252F93537595_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:35:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364517/93537595_hd.jpg not downloaded yet
2026-04-01 07:35:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364517%252F93537592_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:35:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364517/93537592_hd.jpg not downloaded yet
2026-04-01 07:35:32 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: bc95e64b-7b82-5f67-b253-ad765166b866 with identifier: 3365769
2026-04-01 07:35:32 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 07:35:32 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=white+Metallic HTTP/1.1" 200 None
2026-04-01 07:35:32 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 07:35:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 07:35:32 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 07:35:32 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=Electric+vehicle HTTP/1.1" 200 None
2026-04-01 07:35:32 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 07:35:32 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 07:35:32 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: bc95e64b-7b82-5f67-b253-ad765166b866 with identifier: 3365769
2026-04-01 07:35:33 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: bc95e64b-7b82-5f67-b253-ad765166b866 with identifier: 3365769 to the API
2026-04-01 07:35:33 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 07:35:34 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33658/3365769/93608925_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 81347',
'body_type': 'All-terrain vehicle',
'category': 'All-terrain vehicle',
'color': 'white Metallic',
'currency': 'EUR',
'engine_horse_power': 194,
'expiration_date': '2026-04-03T08:00:00Z',
'fuel_type': 'Electric vehicle',
'id': 'bc95e64b-7b82-5f67-b253-ad765166b866',
'is_damaged': False,
'is_operable': True,
'is_vat_included': True,
'link_to_web_offer': 'https://autobid.de/en/item/kia-ev6-gt-4wd-3365769',
'listing': 'auction',
'location': 'Saarbrücken',
'make': 'Kia',
'model': 'EV6 GT',
'odometer': 8200,
'odometer_reading_unit': 'km',
'origin_country_code': 'DE',
'original_photo_count': 5,
'photo_urls': ['https://cdn.autobid.de/data/cars/33658/3365769/93608925_hd.jpg',
'https://cdn.autobid.de/data/cars/33658/3365769/93608919_hd.jpg',
'https://cdn.autobid.de/data/cars/33658/3365769/93608936_hd.jpg',
'https://cdn.autobid.de/data/cars/33658/3365769/93608935_hd.jpg',
'https://cdn.autobid.de/data/cars/33658/3365769/93608933_hd.jpg'],
'photos': [],
'price': 30300,
'price_includes_vat': True,
'registration_date': '2023-12-01',
'seats_number': 5,
'seller_name': 'Subsidiary/authorised dealer of the brand(s): Hyundai, Honda',
'title': 'Kia EV6 GT 4WD',
'transmission': 'automatic',
'vin': None,
'year': 2023}
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365769%27 HTTP/1.1" 200 None
2026-04-01 07:35:34 [autobid.de] INFO: Saving data for 3365769: {'created_time': 1775028934.140629, 'last_price_update_time': 1775028934.140642, 'auction_closing_time': 1775203200.0}
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3365769') HTTP/1.1" 204 0
2026-04-01 07:35:34 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item bc95e64b-7b82-5f67-b253-ad765166b866 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33658/3365769/93608925_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:35:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=93> (referer: None)
2026-04-01 07:35:34 [autobid.de] INFO: Page 93/144: found 20 items
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364523%27 HTTP/1.1" 200 None
2026-04-01 07:35:34 [autobid.de] INFO: Scrape type for 3364523: 1
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364655%27 HTTP/1.1" 200 None
2026-04-01 07:35:34 [autobid.de] INFO: Scrape type for 3364655: 1
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364656%27 HTTP/1.1" 200 None
2026-04-01 07:35:34 [autobid.de] INFO: Scrape type for 3364656: 1
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363373%27 HTTP/1.1" 200 None
2026-04-01 07:35:34 [autobid.de] INFO: Scrape type for 3363373: 1
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363361%27 HTTP/1.1" 200 None
2026-04-01 07:35:34 [autobid.de] INFO: Scrape type for 3363361: 1
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364519%27 HTTP/1.1" 200 None
2026-04-01 07:35:34 [autobid.de] INFO: Scrape type for 3364519: 1
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364522%27 HTTP/1.1" 200 None
2026-04-01 07:35:34 [autobid.de] INFO: Scrape type for 3364522: 1
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364521%27 HTTP/1.1" 200 None
2026-04-01 07:35:34 [autobid.de] INFO: Scrape type for 3364521: 1
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364524%27 HTTP/1.1" 200 None
2026-04-01 07:35:34 [autobid.de] INFO: Scrape type for 3364524: 1
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364624%27 HTTP/1.1" 200 None
2026-04-01 07:35:34 [autobid.de] INFO: Scrape type for 3364624: 1
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364628%27 HTTP/1.1" 200 None
2026-04-01 07:35:34 [autobid.de] INFO: Scrape type for 3364628: 1
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364516%27 HTTP/1.1" 200 None
2026-04-01 07:35:34 [autobid.de] INFO: Scrape type for 3364516: 1
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365487%27 HTTP/1.1" 200 None
2026-04-01 07:35:34 [autobid.de] INFO: Scrape type for 3365487: 1
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363698%27 HTTP/1.1" 200 None
2026-04-01 07:35:34 [autobid.de] INFO: Scrape type for 3363698: 1
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273352516%27 HTTP/1.1" 200 None
2026-04-01 07:35:34 [autobid.de] INFO: Scrape type for 3352516: 1
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354606%27 HTTP/1.1" 200 None
2026-04-01 07:35:34 [autobid.de] INFO: Scrape type for 3354606: 1
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363229%27 HTTP/1.1" 200 None
2026-04-01 07:35:34 [autobid.de] INFO: Scrape type for 3363229: 1
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365488%27 HTTP/1.1" 200 None
2026-04-01 07:35:34 [autobid.de] INFO: Scrape type for 3365488: 1
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364526%27 HTTP/1.1" 200 None
2026-04-01 07:35:34 [autobid.de] INFO: Scrape type for 3364526: 1
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:35:34 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364531%27 HTTP/1.1" 200 None
2026-04-01 07:35:34 [autobid.de] INFO: Scrape type for 3364531: 1
2026-04-01 07:35:42 [scrapy.extensions.logstats] INFO: Crawled 299 pages (at 5 pages/min), scraped 14 items (at 1 items/min)
2026-04-01 07:35:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 261MiB
2026-04-01 07:36:42 [scrapy.extensions.logstats] INFO: Crawled 299 pages (at 0 pages/min), scraped 14 items (at 0 items/min)
2026-04-01 07:36:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 261MiB
2026-04-01 07:37:42 [scrapy.extensions.logstats] INFO: Crawled 299 pages (at 0 pages/min), scraped 14 items (at 0 items/min)
2026-04-01 07:37:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 261MiB
2026-04-01 07:38:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 9bd0ff8c-f433-5d6f-a458-9ccce5b2c4c7 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33657/3365668/93599771_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:38:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 9bd0ff8c-f433-5d6f-a458-9ccce5b2c4c7 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33657/3365668/93599766_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:38:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/nissan-x-trail-1-3-automatik-3364531/details> (referer: None)
2026-04-01 07:38:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 39073ead-8d5c-5e82-953b-e8cef7e692ec with identifier: 3364531
2026-04-01 07:38:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 39073ead-8d5c-5e82-953b-e8cef7e692ec with scrape type 1
2026-04-01 07:38:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 39073ead-8d5c-5e82-953b-e8cef7e692ec
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364531%252F93538088_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364531/93538088_hd.jpg not downloaded yet
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364531%252F93538094_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364531/93538094_hd.jpg not downloaded yet
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364531%252F93538093_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364531/93538093_hd.jpg not downloaded yet
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364531%252F93538100_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364531/93538100_hd.jpg not downloaded yet
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364531%252F93538082_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364531/93538082_hd.jpg not downloaded yet
2026-04-01 07:38:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=94> (referer: None)
2026-04-01 07:38:22 [autobid.de] INFO: Page 94/144: found 20 items
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363581%27 HTTP/1.1" 200 None
2026-04-01 07:38:22 [autobid.de] INFO: Scrape type for 3363581: 1
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363582%27 HTTP/1.1" 200 None
2026-04-01 07:38:22 [autobid.de] INFO: Scrape type for 3363582: 1
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364528%27 HTTP/1.1" 200 None
2026-04-01 07:38:22 [autobid.de] INFO: Scrape type for 3364528: 1
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364529%27 HTTP/1.1" 200 None
2026-04-01 07:38:22 [autobid.de] INFO: Scrape type for 3364529: 1
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364530%27 HTTP/1.1" 200 None
2026-04-01 07:38:22 [autobid.de] INFO: Scrape type for 3364530: 1
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364533%27 HTTP/1.1" 200 None
2026-04-01 07:38:22 [autobid.de] INFO: Scrape type for 3364533: 1
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364659%27 HTTP/1.1" 200 None
2026-04-01 07:38:22 [autobid.de] INFO: Scrape type for 3364659: 1
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364669%27 HTTP/1.1" 200 None
2026-04-01 07:38:22 [autobid.de] INFO: Scrape type for 3364669: 1
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364657%27 HTTP/1.1" 200 None
2026-04-01 07:38:22 [autobid.de] INFO: Scrape type for 3364657: 1
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364661%27 HTTP/1.1" 200 None
2026-04-01 07:38:22 [autobid.de] INFO: Scrape type for 3364661: 1
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364670%27 HTTP/1.1" 200 None
2026-04-01 07:38:22 [autobid.de] INFO: Scrape type for 3364670: 1
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364663%27 HTTP/1.1" 200 None
2026-04-01 07:38:22 [autobid.de] INFO: Scrape type for 3364663: 1
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364555%27 HTTP/1.1" 200 None
2026-04-01 07:38:22 [autobid.de] INFO: Scrape type for 3364555: 1
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364562%27 HTTP/1.1" 200 None
2026-04-01 07:38:22 [autobid.de] INFO: Scrape type for 3364562: 1
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364565%27 HTTP/1.1" 200 None
2026-04-01 07:38:22 [autobid.de] INFO: Scrape type for 3364565: 1
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364556%27 HTTP/1.1" 200 None
2026-04-01 07:38:22 [autobid.de] INFO: Scrape type for 3364556: 1
2026-04-01 07:38:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364564%27 HTTP/1.1" 200 None
2026-04-01 07:38:23 [autobid.de] INFO: Scrape type for 3364564: 1
2026-04-01 07:38:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364557%27 HTTP/1.1" 200 None
2026-04-01 07:38:23 [autobid.de] INFO: Scrape type for 3364557: 1
2026-04-01 07:38:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363583%27 HTTP/1.1" 200 None
2026-04-01 07:38:23 [autobid.de] INFO: Scrape type for 3363583: 1
2026-04-01 07:38:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364559%27 HTTP/1.1" 200 None
2026-04-01 07:38:23 [autobid.de] INFO: Scrape type for 3364559: 1
2026-04-01 07:38:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ford-kuga-1-5-ecoboost-st-line-3364526/details> (referer: None)
2026-04-01 07:38:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 6d545f22-f673-5838-adbb-3cf462ab7cf1 with identifier: 3364526
2026-04-01 07:38:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6d545f22-f673-5838-adbb-3cf462ab7cf1 with scrape type 1
2026-04-01 07:38:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 6d545f22-f673-5838-adbb-3cf462ab7cf1
2026-04-01 07:38:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364526%252F93537900_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364526/93537900_hd.jpg not downloaded yet
2026-04-01 07:38:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364526%252F93537899_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364526/93537899_hd.jpg not downloaded yet
2026-04-01 07:38:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364526%252F93537895_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364526/93537895_hd.jpg not downloaded yet
2026-04-01 07:38:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364526%252F93537906_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364526/93537906_hd.jpg not downloaded yet
2026-04-01 07:38:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364526%252F93537898_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364526/93537898_hd.jpg not downloaded yet
2026-04-01 07:38:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item b037c7a8-46d7-5ee8-92f7-82ca97e2df64 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33656/3365505/93640372_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:38:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item b037c7a8-46d7-5ee8-92f7-82ca97e2df64 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33656/3365505/93640371_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:38:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=95> (referer: None)
2026-04-01 07:38:26 [autobid.de] INFO: Page 95/144: found 20 items
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364571%27 HTTP/1.1" 200 None
2026-04-01 07:38:26 [autobid.de] INFO: Scrape type for 3364571: 1
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364558%27 HTTP/1.1" 200 None
2026-04-01 07:38:26 [autobid.de] INFO: Scrape type for 3364558: 1
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364573%27 HTTP/1.1" 200 None
2026-04-01 07:38:26 [autobid.de] INFO: Scrape type for 3364573: 1
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364560%27 HTTP/1.1" 200 None
2026-04-01 07:38:26 [autobid.de] INFO: Scrape type for 3364560: 1
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364570%27 HTTP/1.1" 200 None
2026-04-01 07:38:26 [autobid.de] INFO: Scrape type for 3364570: 1
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364575%27 HTTP/1.1" 200 None
2026-04-01 07:38:26 [autobid.de] INFO: Scrape type for 3364575: 1
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364561%27 HTTP/1.1" 200 None
2026-04-01 07:38:26 [autobid.de] INFO: Scrape type for 3364561: 1
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363545%27 HTTP/1.1" 200 None
2026-04-01 07:38:26 [autobid.de] INFO: Scrape type for 3363545: 1
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363596%27 HTTP/1.1" 200 None
2026-04-01 07:38:26 [autobid.de] INFO: Scrape type for 3363596: 1
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364578%27 HTTP/1.1" 200 None
2026-04-01 07:38:26 [autobid.de] INFO: Scrape type for 3364578: 1
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364563%27 HTTP/1.1" 200 None
2026-04-01 07:38:26 [autobid.de] INFO: Scrape type for 3364563: 1
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364580%27 HTTP/1.1" 200 None
2026-04-01 07:38:26 [autobid.de] INFO: Scrape type for 3364580: 1
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364567%27 HTTP/1.1" 200 None
2026-04-01 07:38:26 [autobid.de] INFO: Scrape type for 3364567: 1
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364576%27 HTTP/1.1" 200 None
2026-04-01 07:38:26 [autobid.de] INFO: Scrape type for 3364576: 1
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364574%27 HTTP/1.1" 200 None
2026-04-01 07:38:26 [autobid.de] INFO: Scrape type for 3364574: 1
2026-04-01 07:38:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364568%27 HTTP/1.1" 200 None
2026-04-01 07:38:27 [autobid.de] INFO: Scrape type for 3364568: 1
2026-04-01 07:38:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364581%27 HTTP/1.1" 200 None
2026-04-01 07:38:27 [autobid.de] INFO: Scrape type for 3364581: 1
2026-04-01 07:38:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364582%27 HTTP/1.1" 200 None
2026-04-01 07:38:27 [autobid.de] INFO: Scrape type for 3364582: 1
2026-04-01 07:38:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364569%27 HTTP/1.1" 200 None
2026-04-01 07:38:27 [autobid.de] INFO: Scrape type for 3364569: 1
2026-04-01 07:38:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364588%27 HTTP/1.1" 200 None
2026-04-01 07:38:27 [autobid.de] INFO: Scrape type for 3364588: 1
2026-04-01 07:38:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 3fe92428-0159-576f-99a1-15baefeda4fc due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33589/3358842/93464448_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:38:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 3fe92428-0159-576f-99a1-15baefeda4fc due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33589/3358842/93464447_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:38:28 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mercedes-benz-cla-250-e-shooting-brake-8g-dct-amg-line-3364588/details> (referer: None)
2026-04-01 07:38:29 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 2588c5e5-2798-5cb6-99f9-0b406bb26901 with identifier: 3364588
2026-04-01 07:38:29 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 2588c5e5-2798-5cb6-99f9-0b406bb26901 with scrape type 1
2026-04-01 07:38:29 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 2588c5e5-2798-5cb6-99f9-0b406bb26901
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364588%252F93539874_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364588/93539874_hd.jpg not downloaded yet
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364588%252F93539854_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364588/93539854_hd.jpg not downloaded yet
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364588%252F93539872_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364588/93539872_hd.jpg not downloaded yet
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364588%252F93539853_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364588/93539853_hd.jpg not downloaded yet
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364588%252F93539875_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364588/93539875_hd.jpg not downloaded yet
2026-04-01 07:38:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=96> (referer: None)
2026-04-01 07:38:29 [autobid.de] INFO: Page 96/144: found 20 items
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364579%27 HTTP/1.1" 200 None
2026-04-01 07:38:29 [autobid.de] INFO: Scrape type for 3364579: 1
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364583%27 HTTP/1.1" 200 None
2026-04-01 07:38:29 [autobid.de] INFO: Scrape type for 3364583: 1
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364590%27 HTTP/1.1" 200 None
2026-04-01 07:38:29 [autobid.de] INFO: Scrape type for 3364590: 1
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364592%27 HTTP/1.1" 200 None
2026-04-01 07:38:29 [autobid.de] INFO: Scrape type for 3364592: 1
2026-04-01 07:38:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/ford-focus-turnier-1-0-ecoboost-titanium-3364569/details> (referer: None)
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364577%27 HTTP/1.1" 200 None
2026-04-01 07:38:29 [autobid.de] INFO: Scrape type for 3364577: 1
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364586%27 HTTP/1.1" 200 None
2026-04-01 07:38:29 [autobid.de] INFO: Scrape type for 3364586: 1
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364595%27 HTTP/1.1" 200 None
2026-04-01 07:38:29 [autobid.de] INFO: Scrape type for 3364595: 1
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364572%27 HTTP/1.1" 200 None
2026-04-01 07:38:29 [autobid.de] INFO: Scrape type for 3364572: 1
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363594%27 HTTP/1.1" 200 None
2026-04-01 07:38:29 [autobid.de] INFO: Scrape type for 3363594: 1
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363593%27 HTTP/1.1" 200 None
2026-04-01 07:38:29 [autobid.de] INFO: Scrape type for 3363593: 1
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364597%27 HTTP/1.1" 200 None
2026-04-01 07:38:29 [autobid.de] INFO: Scrape type for 3364597: 1
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364633%27 HTTP/1.1" 200 None
2026-04-01 07:38:29 [autobid.de] INFO: Scrape type for 3364633: 1
2026-04-01 07:38:29 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 0e472aaf-139f-5daf-af41-883a4a871ac0 with identifier: 3364569
2026-04-01 07:38:29 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 0e472aaf-139f-5daf-af41-883a4a871ac0 with scrape type 1
2026-04-01 07:38:29 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 0e472aaf-139f-5daf-af41-883a4a871ac0
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364569%252F93539307_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364569/93539307_hd.jpg not downloaded yet
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364596%27 HTTP/1.1" 200 None
2026-04-01 07:38:29 [autobid.de] INFO: Scrape type for 3364596: 1
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364569%252F93539312_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:29 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364569/93539312_hd.jpg not downloaded yet
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:29 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363595%27 HTTP/1.1" 200 None
2026-04-01 07:38:29 [autobid.de] INFO: Scrape type for 3363595: 1
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364569%252F93539319_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364569/93539319_hd.jpg not downloaded yet
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363227%27 HTTP/1.1" 200 None
2026-04-01 07:38:30 [autobid.de] INFO: Scrape type for 3363227: 1
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364569%252F93539316_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364569/93539316_hd.jpg not downloaded yet
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363228%27 HTTP/1.1" 200 None
2026-04-01 07:38:30 [autobid.de] INFO: Scrape type for 3363228: 1
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364569%252F93539323_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364569/93539323_hd.jpg not downloaded yet
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273316241%27 HTTP/1.1" 200 None
2026-04-01 07:38:30 [autobid.de] INFO: Scrape type for 3316241: 1
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273354165%27 HTTP/1.1" 200 None
2026-04-01 07:38:30 [autobid.de] INFO: Scrape type for 3354165: 1
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363697%27 HTTP/1.1" 200 None
2026-04-01 07:38:30 [autobid.de] INFO: Scrape type for 3363697: 1
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363218%27 HTTP/1.1" 200 None
2026-04-01 07:38:30 [autobid.de] INFO: Scrape type for 3363218: 1
2026-04-01 07:38:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 3fe92428-0159-576f-99a1-15baefeda4fc due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33589/3358842/93464446_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:38:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/vw-t-roc-cabriolet-1-5-tsi-act-dsg-r-line-3364592/details> (referer: None)
2026-04-01 07:38:30 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: a5f7ec72-6286-5361-8695-99c896df7ae8 with identifier: 3364592
2026-04-01 07:38:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item a5f7ec72-6286-5361-8695-99c896df7ae8 with scrape type 1
2026-04-01 07:38:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item a5f7ec72-6286-5361-8695-99c896df7ae8
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364592%252F93539988_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364592/93539988_hd.jpg not downloaded yet
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364592%252F93540001_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364592/93540001_hd.jpg not downloaded yet
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364592%252F93539994_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364592/93539994_hd.jpg not downloaded yet
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364592%252F93539986_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364592/93539986_hd.jpg not downloaded yet
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364592%252F93539998_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364592/93539998_hd.jpg not downloaded yet
2026-04-01 07:38:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/opel-corsa-1-5-cdti-edition-3354165/details> (referer: None)
2026-04-01 07:38:31 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 71450a2a-7846-51ce-831a-246869e259b7 with identifier: 3354165
2026-04-01 07:38:31 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 71450a2a-7846-51ce-831a-246869e259b7 with scrape type 1
2026-04-01 07:38:31 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 71450a2a-7846-51ce-831a-246869e259b7
2026-04-01 07:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33542%252F3354165%252F93522979_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33542/3354165/93522979_hd.jpg not downloaded yet
2026-04-01 07:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33542%252F3354165%252F93522980_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33542/3354165/93522980_hd.jpg not downloaded yet
2026-04-01 07:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33542%252F3354165%252F93522981_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33542/3354165/93522981_hd.jpg not downloaded yet
2026-04-01 07:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33542%252F3354165%252F93522982_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33542/3354165/93522982_hd.jpg not downloaded yet
2026-04-01 07:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33542%252F3354165%252F93522983_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:38:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33542/3354165/93522983_hd.jpg not downloaded yet
2026-04-01 07:38:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item 3fe92428-0159-576f-99a1-15baefeda4fc due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33589/3358842/93464445_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:38:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=97> (referer: None)
2026-04-01 07:38:35 [autobid.de] INFO: Page 97/144: found 20 items
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363226%27 HTTP/1.1" 200 None
2026-04-01 07:38:35 [autobid.de] INFO: Scrape type for 3363226: 1
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364585%27 HTTP/1.1" 200 None
2026-04-01 07:38:35 [autobid.de] INFO: Scrape type for 3364585: 1
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364600%27 HTTP/1.1" 200 None
2026-04-01 07:38:35 [autobid.de] INFO: Scrape type for 3364600: 1
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364587%27 HTTP/1.1" 200 None
2026-04-01 07:38:35 [autobid.de] INFO: Scrape type for 3364587: 1
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364589%27 HTTP/1.1" 200 None
2026-04-01 07:38:35 [autobid.de] INFO: Scrape type for 3364589: 1
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364598%27 HTTP/1.1" 200 None
2026-04-01 07:38:35 [autobid.de] INFO: Scrape type for 3364598: 1
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364591%27 HTTP/1.1" 200 None
2026-04-01 07:38:35 [autobid.de] INFO: Scrape type for 3364591: 1
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364599%27 HTTP/1.1" 200 None
2026-04-01 07:38:35 [autobid.de] INFO: Scrape type for 3364599: 1
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364593%27 HTTP/1.1" 200 None
2026-04-01 07:38:35 [autobid.de] INFO: Scrape type for 3364593: 1
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364601%27 HTTP/1.1" 200 None
2026-04-01 07:38:35 [autobid.de] INFO: Scrape type for 3364601: 1
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364549%27 HTTP/1.1" 200 None
2026-04-01 07:38:35 [autobid.de] INFO: Scrape type for 3364549: 1
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364545%27 HTTP/1.1" 200 None
2026-04-01 07:38:35 [autobid.de] INFO: Scrape type for 3364545: 1
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364550%27 HTTP/1.1" 200 None
2026-04-01 07:38:35 [autobid.de] INFO: Scrape type for 3364550: 1
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364546%27 HTTP/1.1" 200 None
2026-04-01 07:38:35 [autobid.de] INFO: Scrape type for 3364546: 1
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364553%27 HTTP/1.1" 200 None
2026-04-01 07:38:35 [autobid.de] INFO: Scrape type for 3364553: 1
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364547%27 HTTP/1.1" 200 None
2026-04-01 07:38:35 [autobid.de] INFO: Scrape type for 3364547: 1
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364552%27 HTTP/1.1" 200 None
2026-04-01 07:38:35 [autobid.de] INFO: Scrape type for 3364552: 1
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364548%27 HTTP/1.1" 200 None
2026-04-01 07:38:35 [autobid.de] INFO: Scrape type for 3364548: 1
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364551%27 HTTP/1.1" 200 None
2026-04-01 07:38:35 [autobid.de] INFO: Scrape type for 3364551: 1
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:38:35 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366889%27 HTTP/1.1" 200 None
2026-04-01 07:38:35 [autobid.de] INFO: Scrape type for 3366889: 1
2026-04-01 07:38:42 [scrapy.extensions.logstats] INFO: Crawled 309 pages (at 10 pages/min), scraped 14 items (at 0 items/min)
2026-04-01 07:38:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 261MiB
2026-04-01 07:39:42 [scrapy.extensions.logstats] INFO: Crawled 309 pages (at 0 pages/min), scraped 14 items (at 0 items/min)
2026-04-01 07:39:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 261MiB
2026-04-01 07:40:42 [scrapy.extensions.logstats] INFO: Crawled 309 pages (at 0 pages/min), scraped 14 items (at 0 items/min)
2026-04-01 07:40:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 261MiB
2026-04-01 07:41:20 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: 3fe92428-0159-576f-99a1-15baefeda4fc with identifier: 3358842
2026-04-01 07:41:20 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 07:41:20 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=silver HTTP/1.1" 200 None
2026-04-01 07:41:20 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 07:41:20 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 07:41:20 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: 3fe92428-0159-576f-99a1-15baefeda4fc with identifier: 3358842
2026-04-01 07:41:20 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: 3fe92428-0159-576f-99a1-15baefeda4fc with identifier: 3358842 to the API
2026-04-01 07:41:20 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 07:41:21 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 07:41:21 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33589/3358842/93461884_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 81347',
'body_type': 'Saloon',
'category': 'Saloon',
'color': 'silver',
'currency': 'EUR',
'emission_standard': 'EURO 6d',
'engine': '1.5L',
'engine_horse_power': 102,
'expiration_date': '2026-04-03T08:00:00Z',
'fuel_type': 'Diesel',
'id': '3fe92428-0159-576f-99a1-15baefeda4fc',
'is_damaged': False,
'is_operable': True,
'is_vat_included': True,
'link_to_web_offer': 'https://autobid.de/en/item/opel-corsa-1-5-cdti-edition-3358842',
'listing': 'auction',
'location': 'Weiterstadt',
'make': 'Opel',
'model': 'Corsa',
'odometer': 59400,
'odometer_reading_unit': 'km',
'origin_country_code': 'DE',
'original_photo_count': 5,
'photo_urls': ['https://cdn.autobid.de/data/cars/33589/3358842/93461884_hd.jpg',
'https://cdn.autobid.de/data/cars/33589/3358842/93464445_hd.jpg',
'https://cdn.autobid.de/data/cars/33589/3358842/93464446_hd.jpg',
'https://cdn.autobid.de/data/cars/33589/3358842/93464447_hd.jpg',
'https://cdn.autobid.de/data/cars/33589/3358842/93464448_hd.jpg'],
'photos': [],
'price': 9900,
'price_includes_vat': True,
'registration_date': '2021-03-01',
'seats_number': 5,
'seller_name': 'Fleet company',
'title': 'Opel Corsa 1,5 CDTI "Edition" DAMAGED BY HAIL',
'transmission': 'manual',
'vin': None,
'year': 2021}
2026-04-01 07:41:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273358842%27 HTTP/1.1" 200 None
2026-04-01 07:41:21 [autobid.de] INFO: Saving data for 3358842: {'created_time': 1775029281.252591, 'last_price_update_time': 1775029281.252604, 'auction_closing_time': 1775203200.0}
2026-04-01 07:41:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:21 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3358842') HTTP/1.1" 204 0
2026-04-01 07:41:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item 3fe92428-0159-576f-99a1-15baefeda4fc due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33589/3358842/93461884_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:41:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item b037c7a8-46d7-5ee8-92f7-82ca97e2df64 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33656/3365505/93640370_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:41:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=98> (referer: None)
2026-04-01 07:41:22 [autobid.de] INFO: Page 98/143: found 20 items
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364540%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [autobid.de] INFO: Scrape type for 3364540: 1
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364541%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [autobid.de] INFO: Scrape type for 3364541: 1
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364542%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [autobid.de] INFO: Scrape type for 3364542: 1
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364543%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [autobid.de] INFO: Scrape type for 3364543: 1
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364544%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [autobid.de] INFO: Scrape type for 3364544: 1
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364525%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [autobid.de] INFO: Scrape type for 3364525: 1
2026-04-01 07:41:22 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mercedes-benz-glb-220d-4matic-8g-dct-amg-line-3366889/details> (referer: None)
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363044%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [autobid.de] INFO: Scrape type for 3363044: 1
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365202%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [autobid.de] INFO: Scrape type for 3365202: 1
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365203%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [autobid.de] INFO: Scrape type for 3365203: 1
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364623%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [autobid.de] INFO: Scrape type for 3364623: 1
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364632%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [autobid.de] INFO: Scrape type for 3364632: 1
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273363342%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [autobid.de] INFO: Scrape type for 3363342: 1
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273364634%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [autobid.de] INFO: Scrape type for 3364634: 1
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365278%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [autobid.de] INFO: Scrape type for 3365278: 1
2026-04-01 07:41:22 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 6c6dd0a5-75e7-5e33-a1fd-2525d7203dab with identifier: 3366889
2026-04-01 07:41:22 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6c6dd0a5-75e7-5e33-a1fd-2525d7203dab with scrape type 1
2026-04-01 07:41:22 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 6c6dd0a5-75e7-5e33-a1fd-2525d7203dab
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33669%252F3366889%252F93648238_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33669/3366889/93648238_hd.jpg not downloaded yet
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366378%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [autobid.de] INFO: Scrape type for 3366378: 1
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33669%252F3366889%252F93648251_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33669/3366889/93648251_hd.jpg not downloaded yet
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365484%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [autobid.de] INFO: Scrape type for 3365484: 1
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33669%252F3366889%252F93648246_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33669/3366889/93648246_hd.jpg not downloaded yet
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365482%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [autobid.de] INFO: Scrape type for 3365482: 1
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33669%252F3366889%252F93648229_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33669/3366889/93648229_hd.jpg not downloaded yet
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366887%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [autobid.de] INFO: Scrape type for 3366887: 1
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33669%252F3366889%252F93648228_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33669/3366889/93648228_hd.jpg not downloaded yet
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366888%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [autobid.de] INFO: Scrape type for 3366888: 1
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:22 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366890%27 HTTP/1.1" 200 None
2026-04-01 07:41:22 [autobid.de] INFO: Scrape type for 3366890: 1
2026-04-01 07:41:23 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/opel-insignia-sports-tourer-1-5-cdti-elegance-3364551/details> (referer: None)
2026-04-01 07:41:23 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: ebcd358e-769a-5a0a-8ddf-e4d146b83d61 with identifier: 3364551
2026-04-01 07:41:23 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item ebcd358e-769a-5a0a-8ddf-e4d146b83d61 with scrape type 1
2026-04-01 07:41:23 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item ebcd358e-769a-5a0a-8ddf-e4d146b83d61
2026-04-01 07:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364551%252F93538743_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364551/93538743_hd.jpg not downloaded yet
2026-04-01 07:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364551%252F93538754_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364551/93538754_hd.jpg not downloaded yet
2026-04-01 07:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364551%252F93538738_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364551/93538738_hd.jpg not downloaded yet
2026-04-01 07:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364551%252F93538755_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364551/93538755_hd.jpg not downloaded yet
2026-04-01 07:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:23 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364551%252F93538753_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:23 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364551/93538753_hd.jpg not downloaded yet
2026-04-01 07:41:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 1 for item b037c7a8-46d7-5ee8-92f7-82ca97e2df64 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33656/3365505/93640369_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:41:24 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/mercedes-benz-eqb-250-progressive-3364525/details> (referer: None)
2026-04-01 07:41:24 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 95e0a039-a2a9-53fa-a21d-0a05426d6bd1 with identifier: 3364525
2026-04-01 07:41:24 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 95e0a039-a2a9-53fa-a21d-0a05426d6bd1 with scrape type 1
2026-04-01 07:41:24 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 95e0a039-a2a9-53fa-a21d-0a05426d6bd1
2026-04-01 07:41:24 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364525%252F93537863_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364525/93537863_hd.jpg not downloaded yet
2026-04-01 07:41:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364525%252F93537871_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364525/93537871_hd.jpg not downloaded yet
2026-04-01 07:41:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364525%252F93537878_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364525/93537878_hd.jpg not downloaded yet
2026-04-01 07:41:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364525%252F93537882_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364525/93537882_hd.jpg not downloaded yet
2026-04-01 07:41:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33646%252F3364525%252F93537864_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:25 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33646/3364525/93537864_hd.jpg not downloaded yet
2026-04-01 07:41:25 [crawlers.pipelines.translation_pipeline] INFO: Spider: autobid.de, Translating item: b037c7a8-46d7-5ee8-92f7-82ca97e2df64 with identifier: 3365505
2026-04-01 07:41:25 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): translate.google.com:443
2026-04-01 07:41:25 [urllib3.connectionpool] DEBUG: https://translate.google.com:443 "GET /m?tl=en&sl=auto&q=black HTTP/1.1" 200 None
2026-04-01 07:41:25 [crawlers.pipelines.translation_pipeline] INFO: Saving translations to Azure Table Storage
2026-04-01 07:41:25 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /Translations(PartitionKey='auto',RowKey='auto') HTTP/1.1" 204 0
2026-04-01 07:41:25 [crawlers.pipelines.item_rules_pipeline] INFO: Spider: autobid.de, Applying rules to item: b037c7a8-46d7-5ee8-92f7-82ca97e2df64 with identifier: 3365505
2026-04-01 07:41:25 [crawlers.pipelines.post_to_api] INFO: Spider: autobid.de, Posting item: b037c7a8-46d7-5ee8-92f7-82ca97e2df64 with identifier: 3365505 to the API
2026-04-01 07:41:25 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): api.app.infinit.cc:443
2026-04-01 07:41:26 [urllib3.connectionpool] DEBUG: https://api.app.infinit.cc:443 "POST /api/command/Alx.Cars.Contracts.Internal.Sourcing.AddListing HTTP/1.1" 200 None
2026-04-01 07:41:26 [scrapy.core.scraper] DEBUG: Scraped from User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33656/3365505/93640368_hd.jpg took longer than 180.0 seconds..
{'auction_title': 'Auction 81347',
'body_type': 'Saloon',
'category': 'Saloon',
'color': 'black',
'currency': 'EUR',
'emission_standard': 'EURO 6d-Temp',
'engine': '1.5L',
'engine_horse_power': 140,
'expiration_date': '2026-04-03T08:00:00Z',
'fuel_type': 'Petrol',
'id': 'b037c7a8-46d7-5ee8-92f7-82ca97e2df64',
'is_damaged': True,
'is_operable': True,
'is_vat_deductible': False,
'link_to_web_offer': 'https://autobid.de/en/item/bmw-118i-advantage-3365505',
'listing': 'auction',
'location': 'Baunatal',
'make': 'BMW',
'model': '118i',
'odometer': 106300,
'odometer_reading_unit': 'km',
'origin_country_code': 'DE',
'original_photo_count': 5,
'photo_urls': ['https://cdn.autobid.de/data/cars/33656/3365505/93640368_hd.jpg',
'https://cdn.autobid.de/data/cars/33656/3365505/93640369_hd.jpg',
'https://cdn.autobid.de/data/cars/33656/3365505/93640370_hd.jpg',
'https://cdn.autobid.de/data/cars/33656/3365505/93640371_hd.jpg',
'https://cdn.autobid.de/data/cars/33656/3365505/93640372_hd.jpg'],
'photos': [],
'price': 9800,
'price_includes_vat': False,
'registration_date': '2019-12-01',
'seats_number': 5,
'seller_name': None,
'title': 'BMW 118i "Advantage"',
'transmission': 'manual',
'vin': None,
'year': 2019}
2026-04-01 07:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365505%27 HTTP/1.1" 200 None
2026-04-01 07:41:26 [autobid.de] INFO: Saving data for 3365505: {'created_time': 1775029286.282231, 'last_price_update_time': 1775029286.282242, 'auction_closing_time': 1775203200.0}
2026-04-01 07:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:26 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='autobid.de',RowKey='3365505') HTTP/1.1" 204 0
2026-04-01 07:41:26 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 0 for item b037c7a8-46d7-5ee8-92f7-82ca97e2df64 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33656/3365505/93640368_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:41:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 1fe1caae-b816-5136-b54b-bd539219d9dd due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33638/3363742/93537361_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:41:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=99> (referer: None)
2026-04-01 07:41:27 [autobid.de] INFO: Page 99/143: found 20 items
2026-04-01 07:41:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366877%27 HTTP/1.1" 200 None
2026-04-01 07:41:27 [autobid.de] INFO: Scrape type for 3366877: 1
2026-04-01 07:41:27 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366878%27 HTTP/1.1" 200 None
2026-04-01 07:41:28 [autobid.de] INFO: Scrape type for 3366878: 1
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366879%27 HTTP/1.1" 200 None
2026-04-01 07:41:28 [autobid.de] INFO: Scrape type for 3366879: 1
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366880%27 HTTP/1.1" 200 None
2026-04-01 07:41:28 [autobid.de] INFO: Scrape type for 3366880: 1
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366881%27 HTTP/1.1" 200 None
2026-04-01 07:41:28 [autobid.de] INFO: Scrape type for 3366881: 1
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366882%27 HTTP/1.1" 200 None
2026-04-01 07:41:28 [autobid.de] INFO: Scrape type for 3366882: 1
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366883%27 HTTP/1.1" 200 None
2026-04-01 07:41:28 [autobid.de] INFO: Scrape type for 3366883: 1
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366871%27 HTTP/1.1" 200 None
2026-04-01 07:41:28 [autobid.de] INFO: Scrape type for 3366871: 1
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366872%27 HTTP/1.1" 200 None
2026-04-01 07:41:28 [autobid.de] INFO: Scrape type for 3366872: 1
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366873%27 HTTP/1.1" 200 None
2026-04-01 07:41:28 [autobid.de] INFO: Scrape type for 3366873: 1
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366874%27 HTTP/1.1" 200 None
2026-04-01 07:41:28 [autobid.de] INFO: Scrape type for 3366874: 1
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366875%27 HTTP/1.1" 200 None
2026-04-01 07:41:28 [autobid.de] INFO: Scrape type for 3366875: 1
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366876%27 HTTP/1.1" 200 None
2026-04-01 07:41:28 [autobid.de] INFO: Scrape type for 3366876: 1
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366884%27 HTTP/1.1" 200 None
2026-04-01 07:41:28 [autobid.de] INFO: Scrape type for 3366884: 1
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366885%27 HTTP/1.1" 200 None
2026-04-01 07:41:28 [autobid.de] INFO: Scrape type for 3366885: 1
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273366886%27 HTTP/1.1" 200 None
2026-04-01 07:41:28 [autobid.de] INFO: Scrape type for 3366886: 1
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365485%27 HTTP/1.1" 200 None
2026-04-01 07:41:28 [autobid.de] INFO: Scrape type for 3365485: 1
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365597%27 HTTP/1.1" 200 None
2026-04-01 07:41:28 [autobid.de] INFO: Scrape type for 3365597: 1
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365563%27 HTTP/1.1" 200 None
2026-04-01 07:41:28 [autobid.de] INFO: Scrape type for 3365563: 1
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:28 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365665%27 HTTP/1.1" 200 None
2026-04-01 07:41:28 [autobid.de] INFO: Scrape type for 3365665: 1
2026-04-01 07:41:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 50a6746d-8840-5a0b-9dcb-616de15a4322 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33638/3363741/93537305_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:41:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 39d00a86-abd5-5d7e-ac06-d16494325f00 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33646/3364517/93537592_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:41:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/skoda-octavia-combi-1-9-tdi-ambiente-3365665/details> (referer: None)
2026-04-01 07:41:30 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 7789f583-0415-5baa-a072-f6373f52e096 with identifier: 3365665
2026-04-01 07:41:30 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 7789f583-0415-5baa-a072-f6373f52e096 with scrape type 1
2026-04-01 07:41:30 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 7789f583-0415-5baa-a072-f6373f52e096
2026-04-01 07:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33657%252F3365665%252F93599668_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33657/3365665/93599668_hd.jpg not downloaded yet
2026-04-01 07:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33657%252F3365665%252F93599673_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33657/3365665/93599673_hd.jpg not downloaded yet
2026-04-01 07:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33657%252F3365665%252F93599654_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33657/3365665/93599654_hd.jpg not downloaded yet
2026-04-01 07:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33657%252F3365665%252F93599659_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33657/3365665/93599659_hd.jpg not downloaded yet
2026-04-01 07:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:30 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33657%252F3365665%252F93599662_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:30 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33657/3365665/93599662_hd.jpg not downloaded yet
2026-04-01 07:41:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/search-results?sortingType=auctionStartDate-ASCENDING¤tPage=100> (referer: None)
2026-04-01 07:41:31 [autobid.de] INFO: Page 100/143: found 20 items
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365670%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [autobid.de] INFO: Scrape type for 3365670: 1
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365673%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [autobid.de] INFO: Scrape type for 3365673: 1
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365674%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [autobid.de] INFO: Scrape type for 3365674: 1
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365669%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [autobid.de] INFO: Scrape type for 3365669: 1
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365677%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [autobid.de] INFO: Scrape type for 3365677: 1
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365697%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [autobid.de] INFO: Scrape type for 3365697: 1
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365691%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [autobid.de] INFO: Scrape type for 3365691: 1
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365703%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [autobid.de] INFO: Scrape type for 3365703: 1
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365704%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [autobid.de] INFO: Scrape type for 3365704: 1
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365692%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [autobid.de] INFO: Scrape type for 3365692: 1
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365772%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [autobid.de] INFO: Scrape type for 3365772: 1
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365693%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [autobid.de] INFO: Scrape type for 3365693: 1
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365696%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [autobid.de] INFO: Scrape type for 3365696: 1
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365694%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [autobid.de] INFO: Scrape type for 3365694: 1
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365776%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [autobid.de] INFO: Scrape type for 3365776: 1
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365695%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [autobid.de] INFO: Scrape type for 3365695: 1
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365699%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [autobid.de] INFO: Scrape type for 3365699: 1
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365698%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [autobid.de] INFO: Scrape type for 3365698: 1
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365777%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [autobid.de] INFO: Scrape type for 3365777: 1
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%273365779%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [autobid.de] INFO: Scrape type for 3365779: 1
2026-04-01 07:41:31 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/bmw-430i-cabrio-sport-automatic-m-sport-3365563/details> (referer: None)
2026-04-01 07:41:31 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: 6700e898-8499-5fa1-a35f-c2b493ad8256 with identifier: 3365563
2026-04-01 07:41:31 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item 6700e898-8499-5fa1-a35f-c2b493ad8256 with scrape type 1
2026-04-01 07:41:31 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item 6700e898-8499-5fa1-a35f-c2b493ad8256
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365563%252F93585607_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365563/93585607_hd.jpg not downloaded yet
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365563%252F93585625_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365563/93585625_hd.jpg not downloaded yet
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365563%252F93585614_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365563/93585614_hd.jpg not downloaded yet
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365563%252F93585629_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365563/93585629_hd.jpg not downloaded yet
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:31 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33656%252F3365563%252F93585615_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:31 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33656/3365563/93585615_hd.jpg not downloaded yet
2026-04-01 07:41:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 39d00a86-abd5-5d7e-ac06-d16494325f00 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33646/3364517/93537595_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:41:32 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://autobid.de/en/item/skoda-octavia-combi-1-6-tdi-dsg-style-3365779/details> (referer: None)
2026-04-01 07:41:33 [crawlers.middlewares.id_gen_middleware] INFO: Generated ID for item: e363b07e-ed8f-5a90-96db-d5e4c3b3c526 with identifier: 3365779
2026-04-01 07:41:33 [crawlers.middlewares.report_download_middleware] INFO: Skipping car_expert_report download for item e363b07e-ed8f-5a90-96db-d5e4c3b3c526 with scrape type 1
2026-04-01 07:41:33 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 5 photos for item e363b07e-ed8f-5a90-96db-d5e4c3b3c526
2026-04-01 07:41:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33658%252F3365779%252F93610230_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33658/3365779/93610230_hd.jpg not downloaded yet
2026-04-01 07:41:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33658%252F3365779%252F93610209_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33658/3365779/93610209_hd.jpg not downloaded yet
2026-04-01 07:41:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33658%252F3365779%252F93610211_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33658/3365779/93610211_hd.jpg not downloaded yet
2026-04-01 07:41:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33658%252F3365779%252F93610210_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33658/3365779/93610210_hd.jpg not downloaded yet
2026-04-01 07:41:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-04-01 07:41:33 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27autobid.de%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fcdn.autobid.de%252Fdata%252Fcars%252F33658%252F3365779%252F93610221_hd.jpg%27 HTTP/1.1" 200 None
2026-04-01 07:41:33 [crawlers.middlewares.photo_download_middleware] INFO: Image https://cdn.autobid.de/data/cars/33658/3365779/93610221_hd.jpg not downloaded yet
2026-04-01 07:41:42 [scrapy.extensions.logstats] INFO: Crawled 318 pages (at 9 pages/min), scraped 16 items (at 2 items/min)
2026-04-01 07:41:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 261MiB
2026-04-01 07:42:42 [scrapy.extensions.logstats] INFO: Crawled 318 pages (at 0 pages/min), scraped 16 items (at 0 items/min)
2026-04-01 07:42:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 261MiB
2026-04-01 07:43:42 [scrapy.extensions.logstats] INFO: Crawled 318 pages (at 0 pages/min), scraped 16 items (at 0 items/min)
2026-04-01 07:43:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 261MiB
2026-04-01 07:44:20 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 39073ead-8d5c-5e82-953b-e8cef7e692ec due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33646/3364531/93538082_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:44:21 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 6d545f22-f673-5838-adbb-3cf462ab7cf1 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33646/3364526/93537898_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:44:24 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 6d545f22-f673-5838-adbb-3cf462ab7cf1 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33646/3364526/93537906_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:44:25 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 2 for item 6d545f22-f673-5838-adbb-3cf462ab7cf1 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33646/3364526/93537895_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:44:27 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 2588c5e5-2798-5cb6-99f9-0b406bb26901 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33646/3364588/93539875_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:44:28 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item a5f7ec72-6286-5361-8695-99c896df7ae8 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33646/3364592/93539998_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:44:30 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 4 for item 71450a2a-7846-51ce-831a-246869e259b7 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33542/3354165/93522983_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:44:32 [crawlers.middlewares.photo_download_middleware] DEBUG: Failed to download image 3 for item 71450a2a-7846-51ce-831a-246869e259b7 due to User timeout caused connection failure: Getting https://cdn.autobid.de/data/cars/33542/3354165/93522982_hd.jpg took longer than 180.0 seconds..
2026-04-01 07:44:42 [scrapy.extensions.logstats] INFO: Crawled 318 pages (at 0 pages/min), scraped 16 items (at 0 items/min)
2026-04-01 07:44:42 [scrapy.extensions.memusage] INFO: Peak memory usage is 261MiB
2026-04-01 07:45:23 [scrapy.crawler] INFO: Received SIGTERM, shutting down gracefully. Send again to force
2026-04-01 07:45:23 [scrapy.core.engine] INFO: Closing spider (shutdown)
2026-04-01 07:45:25 [scrapy.crawler] INFO: Received SIGTERM twice, forcing unclean shutdown