-
Head
2026-03-31 21:47:06 [scrapy.utils.log] INFO: Scrapy 2.11.2 started (bot: SourcingV2)
2026-03-31 21:47:06 [scrapy.utils.log] INFO: Versions: lxml 5.2.2.0, libxml2 2.12.6, cssselect 1.2.0, parsel 1.9.1, w3lib 2.1.2, Twisted 24.3.0, Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0], pyOpenSSL 24.1.0 (OpenSSL 3.2.1 30 Jan 2024), cryptography 42.0.7, Platform Linux-5.15.0-1098-azure-x86_64-with-glibc2.36
2026-03-31 21:47:06 [auto_selling_leboncoin.fr] INFO: Starting auto_selling_leboncoin.fr spider
2026-03-31 21:47:06 [auto_selling_leboncoin.fr] INFO: _job: 7b23d4662ce811f19724def17024dec4, SCRAPY_JOB: None
2026-03-31 21:47:06 [scrapy.addons] INFO: Enabled addons:
[]
2026-03-31 21:47:06 [asyncio] DEBUG: Using selector: EpollSelector
2026-03-31 21:47:06 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.asyncioreactor.AsyncioSelectorReactor
2026-03-31 21:47:06 [scrapy.utils.log] DEBUG: Using asyncio event loop: asyncio.unix_events._UnixSelectorEventLoop
2026-03-31 21:47:06 [scrapy.extensions.telnet] INFO: Telnet Password: ffe2b0fc928832ae
2026-03-31 21:47:06 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
'scrapy.extensions.telnet.TelnetConsole',
'scrapy.extensions.memusage.MemoryUsage',
'scrapy.extensions.feedexport.FeedExporter',
'scrapy.extensions.logstats.LogStats',
'scrapy.extensions.closespider.CloseSpider']
2026-03-31 21:47:06 [scrapy.crawler] INFO: Overridden settings:
{'BOT_NAME': 'SourcingV2',
'CLOSESPIDER_TIMEOUT': 3600,
'CONCURRENT_REQUESTS': 5,
'DOWNLOAD_MAXSIZE': 52428800,
'DOWNLOAD_WARNSIZE': 10485760,
'FEED_EXPORT_ENCODING': 'utf-8',
'LOG_FILE': '/var/log/scrapyd/logs/sourcing_v2/auto_selling_leboncoin.fr/7b23d4662ce811f19724def17024dec4.log',
'LOG_FORMATTER': 'crawlers.log_formatter.SourcingLogFormatter',
'MEMUSAGE_LIMIT_MB': 2048,
'MEMUSAGE_WARNING_MB': 1536,
'NEWSPIDER_MODULE': 'spiders',
'REQUEST_FINGERPRINTER_CLASS': 'scrapy_zyte_api.ScrapyZyteAPIRequestFingerprinter',
'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7',
'SPIDER_MODULES': ['spiders', 'auth_check'],
'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor',
'USER_AGENT': ''}
2026-03-31 21:47:06 [scrapy_zyte_api.handler] INFO: Using a Zyte API key starting with '5857011'
2026-03-31 21:47:06 [scrapy_zyte_api.handler] INFO: Using a Zyte API key starting with '5857011'
2026-03-31 21:47:06 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.offsite.OffsiteMiddleware',
'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
'scrapy_zyte_api.ScrapyZyteAPIDownloaderMiddleware',
'scrapy.downloadermiddlewares.retry.RetryMiddleware',
'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
'scrapy.downloadermiddlewares.stats.DownloaderStats']
2026-03-31 21:47:06 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Setting up IdGenerationMiddleware for auto_selling
2026-03-31 21:47:06 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
'scrapy_zyte_api.ScrapyZyteAPISpiderMiddleware',
'scrapy.spidermiddlewares.referer.RefererMiddleware',
'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
'scrapy.spidermiddlewares.depth.DepthMiddleware',
'crawlers.middlewares.photo_download_middleware.PhotoDownloadMiddleware',
'crawlers.auto_selling.middlewares.id_gen_middleware.IdGenMiddleware']
2026-03-31 21:47:06 [scrapy.middleware] INFO: Enabled item pipelines:
['crawlers.pipelines.post_to_azurebus_auto_selling.AzureBusPipelineAutoSelling',
'crawlers.pipelines.save_to_cdlq_db_auto_selling.SaveToCdlqDbAutoSelling']
2026-03-31 21:47:06 [scrapy.core.engine] INFO: Spider opened
2026-03-31 21:47:06 [azure.identity._credentials.environment] INFO: Incomplete environment configuration for EnvironmentCredential. These variables are set: AZURE_CLIENT_ID, AZURE_TENANT_ID
2026-03-31 21:47:06 [common.service_bus.service_bus_client_factory] INFO: ServiceBusClient initialized via DefaultAzureCredential for alx-production-bus.servicebus.windows.net
2026-03-31 21:47:06 [crawlers.pipelines.base_azure_bus_pipeline] INFO: [AzureBusPipeline] Connected to topic auto.sourcing-v2
2026-03-31 21:47:06 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-03-31 21:47:06 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6024
2026-03-31 21:47:06 [scrapy.extensions.memusage] INFO: Peak memory usage is 142MiB
2026-03-31 21:47:06 [scrapy-playwright] INFO: Starting download handler
2026-03-31 21:47:06 [scrapy-playwright] INFO: Starting download handler
2026-03-31 21:47:11 [auto_selling_leboncoin.fr] INFO: Scraper started with URL: https://www.leboncoin.fr/boutique/6756166/asteria.htm and multipost_profile_id: 66429dc6-7db5-40b7-a082-85c6bacd556c and job_id: 7b23d4662ce811f19724def17024dec4
2026-03-31 21:47:11 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 1st time calling it.
2026-03-31 21:47:13 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.leboncoin.fr/boutique/6756166/asteria.htm> (referer: https://www.leboncoin.fr/boutique/6756166/asteria.htm) ['zyte-api']
2026-03-31 21:47:13 [auto_selling_leboncoin.fr] INFO: Captured Zyte cookies: [{'name': '__Secure-Install', 'value': '50fa9140-caf5-4a4a-863a-b6ea0fa11ac9', 'domain': '.leboncoin.fr', 'path': '/', 'expires': 253402300798, 'httpOnly': False, 'secure': True}, {'name': 'cnfdVisitorId', 'value': '474c3df9-5936-4f6f-ae94-6a43eefc2b52', 'domain': '.leboncoin.fr', 'path': '/', 'expires': 1932673632, 'httpOnly': False, 'secure': True}, {'name': 'datadome', 'value': 'v_2HoX_IGXqKech4PgH95xGjaOlkHPhHbmomlz4QzaWs8W_tTG~Rd1rkgjjct_uFQSG4c4OXLfyVJDBADLnz1VmjPAPOzPj24aYi8gexLLtExSY6XINyb0ZYiq1pk9jC', 'domain': '.leboncoin.fr', 'path': '/', 'expires': 1806529632, 'httpOnly': False, 'secure': True}]
2026-03-31 21:47:13 [auto_selling_leboncoin.fr] INFO: Scrapy-formatted cookies: [{'name': '__Secure-Install', 'value': '50fa9140-caf5-4a4a-863a-b6ea0fa11ac9', 'domain': '.leboncoin.fr', 'path': '/'}, {'name': 'cnfdVisitorId', 'value': '474c3df9-5936-4f6f-ae94-6a43eefc2b52', 'domain': '.leboncoin.fr', 'path': '/'}, {'name': 'datadome', 'value': 'v_2HoX_IGXqKech4PgH95xGjaOlkHPhHbmomlz4QzaWs8W_tTG~Rd1rkgjjct_uFQSG4c4OXLfyVJDBADLnz1VmjPAPOzPj24aYi8gexLLtExSY6XINyb0ZYiq1pk9jC', 'domain': '.leboncoin.fr', 'path': '/'}]
2026-03-31 21:47:13 [auto_selling_leboncoin.fr] INFO: Response headers after parse_dealer_home_page: {b'X-Amz-Cf-Pop': [b'CDG50-P2'], b'Date': [b'Tue, 31 Mar 2026 21:47:12 GMT'], b'Vary': [b'Accept-Encoding'], b'Accept-Ch': [b'Sec-CH-UA,Sec-CH-UA-Mobile,Sec-CH-UA-Platform,Sec-CH-UA-Arch,Sec-CH-UA-Full-Version-List,Sec-CH-UA-Model,Sec-CH-Device-Memory'], b'Strict-Transport-Security': [b'max-age=15768000'], b'Via': [b'1.1 2ba5677785db2f66bc73820b2a261476.cloudfront.net (CloudFront)'], b'Set-Cookie': [b'datadome=v_2HoX_IGXqKech4PgH95xGjaOlkHPhHbmomlz4QzaWs8W_tTG~Rd1rkgjjct_uFQSG4c4OXLfyVJDBADLnz1VmjPAPOzPj24aYi8gexLLtExSY6XINyb0ZYiq1pk9jC; Max-Age=31536000; Domain=.leboncoin.fr; Path=/; Secure; SameSite=Lax'], b'Content-Security-Policy': [b'frame-ancestors *.leboncoin.fr *.leboncoin.io *.leboncoin.ci; report-uri https://api.leboncoin.fr/api/csp-report/v1/report/;'], b'Referrer-Policy': [b'no-referrer-when-downgrade'], b'Content-Security-Policy-Report-Only': [b'object-src *.leboncoin.fr *.leboncoin.io *.leboncoin.ci; frame-ancestors *.leboncoin.fr *.leboncoin.io *.leboncoin.ci; report-uri https://api.leboncoin.fr/api/csp-report/v1/report/;'], b'Etag': [b'W/"hwkbvogojubojj"'], b'Content-Type': [b'text/html; charset=utf-8'], b'X-Cache': [b'Miss from cloudfront'], b'X-Datadome': [b'protected'], b'Cache-Control': [b'private, no-cache, no-store, max-age=0, must-revalidate'], b'X-Amz-Cf-Id': [b'oddmzRolglkyvgjCDYxZAhqJ7YzDm_gHAZViLzSndhqTDsKGC-E3zQ==']}
2026-03-31 21:47:13 [auto_selling_leboncoin.fr] INFO: defaultSearchOptions: {'limit': 10, 'filters': {'owner': {'user_id': '2b14a55a-75e0-4e6f-a636-59df95d34026'}, 'enums': {'ad_type': ['offer']}, 'category': {'id': '1'}}}
2026-03-31 21:47:13 [auto_selling_leboncoin.fr] INFO: cookies: [{'name': '__Secure-Install', 'value': '50fa9140-caf5-4a4a-863a-b6ea0fa11ac9', 'domain': '.leboncoin.fr', 'path': '/'}, {'name': 'cnfdVisitorId', 'value': '474c3df9-5936-4f6f-ae94-6a43eefc2b52', 'domain': '.leboncoin.fr', 'path': '/'}, {'name': 'datadome', 'value': 'v_2HoX_IGXqKech4PgH95xGjaOlkHPhHbmomlz4QzaWs8W_tTG~Rd1rkgjjct_uFQSG4c4OXLfyVJDBADLnz1VmjPAPOzPj24aYi8gexLLtExSY6XINyb0ZYiq1pk9jC', 'domain': '.leboncoin.fr', 'path': '/'}]
2026-03-31 21:47:13 [auto_selling_leboncoin.fr] INFO: Payload for get listings: {'limit': 200, 'filters': {'owner': {'user_id': '2b14a55a-75e0-4e6f-a636-59df95d34026'}, 'enums': {'ad_type': ['offer']}, 'category': {'id': '1'}}, 'offset': 0}
2026-03-31 21:47:13 [auto_selling_leboncoin.fr] INFO: Request: curl -X POST https://api.leboncoin.fr/finder/search --data-raw '{"filters": {"category": {"id": "1"}, "enums": {"ad_type": ["offer"]}, "owner": {"user_id": "2b14a55a-75e0-4e6f-a636-59df95d34026"}}, "limit": 200, "offset": 0}' -H 'Accept: */*' -H 'Accept-Language: en-GB,en-US;q=0.9,en;q=0.8' -H 'Content-Type: application/json' -H 'Origin: https://www.leboncoin.fr' --cookie 'name=__Secure-Install; name=cnfdVisitorId; name=datadome'
2026-03-31 21:47:13 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 1st time calling it.
2026-03-31 21:47:14 [scrapy.core.engine] DEBUG: Crawled (200) <POST https://api.leboncoin.fr/finder/search> (referer: https://www.leboncoin.fr/boutique/6756166/asteria.htm) ['zyte-api']
2026-03-31 21:47:14 [auto_selling_leboncoin.fr] INFO: Found 11 listings on page 1
2026-03-31 21:47:14 [auto_selling_leboncoin.fr] INFO: Found listing with ID: 3167911072
2026-03-31 21:47:14 [azure.identity._credentials.environment] INFO: Incomplete environment configuration for EnvironmentCredential. These variables are set: AZURE_CLIENT_ID, AZURE_TENANT_ID
2026-03-31 21:47:14 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2026-03-31 21:47:14 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 2121
2026-03-31 21:47:14 [azure.identity._credentials.chained] INFO: DefaultAzureCredential acquired a token from WorkloadIdentityCredential
2026-03-31 21:47:14 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxsourcingstorageprod.table.core.windows.net:443
2026-03-31 21:47:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-31 21:47:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_leboncoin.fr%27%20and%20RowKey%20eq%20%273167911072%27 HTTP/1.1" 200 None
2026-03-31 21:47:14 [auto_selling_leboncoin.fr] INFO: Scrape type for 3167911072: 1
2026-03-31 21:47:14 [auto_selling_leboncoin.fr] INFO: Scraping new listing 3167911072
2026-03-31 21:47:14 [auto_selling_leboncoin.fr] INFO: Extracted item: {'auction_title': '',
'c_o_2_emission_value': None,
'category': None,
'color': 'Gris',
'currency': 'EUR',
'description': 'Peugeot 3008 Nouveau Hybrid 145 e-DCS6 finition Allure, mise '
-
Tail
'Banquette arrière rabattable 40/20/40\n'
'Vitres électriques et teintées\n'
'Rétroviseurs électriques, dégivrants et rabattables\n'
'Rétroviseurs électrochromes avec mémoire\n'
'Éclairage d’ambiance\n'
'Barres de toit\n'
'Intérieur cuir avec inserts métal\n'
'\n'
'Multimédia & technologie\n'
'\n'
'Écran tactile multifonction couleur\n'
'GPS cartographique\n'
'Bluetooth mains libres\n'
'Commandes vocales\n'
'Radio / CD / DVD / MP3\n'
'Disque dur multimédia\n'
'USB / Prise auxiliaire / Prise 12V\n'
'Ordinateur de bord\n'
'Services connectés / Webradio\n'
'\n'
'Sécurité\n'
'\n'
'ABS / ESP / Antipatinage\n'
'Airbags frontaux, latéraux et rideaux\n'
'Freinage automatique d’urgence\n'
'Système de prévention des collisions\n'
'AFIL\n'
'Contrôle de pression des pneus\n'
'Isofix arrière\n'
'Alarme périmétrique et volumétrique\n'
'\n'
'Prix volontairement attractif tenant compte de quelques '
'défauts esthétiques de carrosserie ainsi que d’un décollement '
'partiel du cuir sur le tableau de bord. Véhicule sain et '
'mécaniquement irréprochable.\n'
'\n'
'Véhicule garanti 3 mois\n'
'Extension de garantie possible selon demande\n'
'Reprise possible selon conditions\n'
'\n'
'Pour plus d’informations ou pour organiser une visite / '
'essai, n’hésitez pas à me contacter.\n'
'Disponibilité immédiate.\n'
'\n'
'Modèles similaires :\n'
'Q7, GLE, Touareg, XC90, Range Rover Sport, Cayenne\n'
'\n'
'Contactez-moi pour plus d’infos ou photos.',
'doors_number': 5,
'emission_standard': None,
'engine': None,
'engine_horse_power': 258,
'expiration_date': None,
'fuel_type': 'Diesel',
'id': 'e776236f-5695-5b77-a9bd-001d96be2390',
'is_damaged': False,
'is_operable': True,
'license_plate': None,
'link_to_web_offer': 'https://www.leboncoin.fr/ad/voitures/3140136769',
'listing': 'auction',
'make': 'BMW',
'model': 'X5',
'odometer': 112500,
'odometer_reading_unit': 'km',
'origin_country_code': 'FR',
'original_photo_count': 5,
'photo_urls': ['https://img.leboncoin.fr/api/v1/lbcpb1/images/58/0a/7c/580a7cd16e83a0e53c1c781c00cec4d8b6e9e1ee.jpg?rule=ad-large',
'https://img.leboncoin.fr/api/v1/lbcpb1/images/9f/5b/eb/9f5bebcc7b2d6a32116dd6f626fffc4bee085d66.jpg?rule=ad-large',
'https://img.leboncoin.fr/api/v1/lbcpb1/images/e5/1c/c8/e51cc8bad44c65806c54b816dc14cbc74eae65f3.jpg?rule=ad-large',
'https://img.leboncoin.fr/api/v1/lbcpb1/images/b5/60/a0/b560a01fba0f6923747834314b0c629d244aef63.jpg?rule=ad-large',
'https://img.leboncoin.fr/api/v1/lbcpb1/images/75/c4/cf/75c4cf79f99a1978f065cf31af3bb7d82737b6ee.jpg?rule=ad-large'],
'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_e776236f-5695-5b77-a9bd-001d96be2390/e776236f-5695-5b77-a9bd-001d96be2390_image_0.jpg',
'Extension': 'jpg',
'FileSizeInBytes': 254391,
'Height': 748,
'OriginUrl': 'https://img.leboncoin.fr/api/v1/lbcpb1/images/58/0a/7c/580a7cd16e83a0e53c1c781c00cec4d8b6e9e1ee.jpg?rule=ad-large',
'Width': 1200},
{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_e776236f-5695-5b77-a9bd-001d96be2390/e776236f-5695-5b77-a9bd-001d96be2390_image_1.jpg',
'Extension': 'jpg',
'FileSizeInBytes': 273781,
'Height': 800,
'OriginUrl': 'https://img.leboncoin.fr/api/v1/lbcpb1/images/9f/5b/eb/9f5bebcc7b2d6a32116dd6f626fffc4bee085d66.jpg?rule=ad-large',
'Width': 1067},
{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_e776236f-5695-5b77-a9bd-001d96be2390/e776236f-5695-5b77-a9bd-001d96be2390_image_2.jpg',
'Extension': 'jpg',
'FileSizeInBytes': 135138,
'Height': 800,
'OriginUrl': 'https://img.leboncoin.fr/api/v1/lbcpb1/images/e5/1c/c8/e51cc8bad44c65806c54b816dc14cbc74eae65f3.jpg?rule=ad-large',
'Width': 1067},
{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_e776236f-5695-5b77-a9bd-001d96be2390/e776236f-5695-5b77-a9bd-001d96be2390_image_3.jpg',
'Extension': 'jpg',
'FileSizeInBytes': 98403,
'Height': 800,
'OriginUrl': 'https://img.leboncoin.fr/api/v1/lbcpb1/images/b5/60/a0/b560a01fba0f6923747834314b0c629d244aef63.jpg?rule=ad-large',
'Width': 600},
{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_e776236f-5695-5b77-a9bd-001d96be2390/e776236f-5695-5b77-a9bd-001d96be2390_image_4.jpg',
'Extension': 'jpg',
'FileSizeInBytes': 97336,
'Height': 800,
'OriginUrl': 'https://img.leboncoin.fr/api/v1/lbcpb1/images/75/c4/cf/75c4cf79f99a1978f065cf31af3bb7d82737b6ee.jpg?rule=ad-large',
'Width': 600}],
'price': 29990,
'price_includes_vat': False,
'registration_date': '2017-06-01',
'seats_number': 5,
'seller_name': None,
'title': 'X5 XDRIVE30DA 258cv XLINE PACK M BVA + Toit ouvrant',
'transmission': 'automatic',
'trim': 'X5 xDrive30dA 258ch xLine',
'vin': None,
'year': 2017}
2026-03-31 21:47:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-31 21:47:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_leboncoin.fr%27%20and%20RowKey%20eq%20%273140136769%27 HTTP/1.1" 200 None
2026-03-31 21:47:17 [auto_selling_leboncoin.fr] INFO: Saving data for 3140136769: {'created_time': 1774993637.479424, 'last_price_update_time': 1774993637.479432}
2026-03-31 21:47:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-31 21:47:17 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_leboncoin.fr',RowKey='3140136769') HTTP/1.1" 204 0
2026-03-31 21:47:17 [auto_selling_leboncoin.fr] INFO: No more listings found to scrape
2026-03-31 21:47:17 [scrapy.core.engine] INFO: Closing spider (finished)
2026-03-31 21:47:17 [crawlers.pipelines.save_to_cdlq_db_auto_selling] DEBUG: [SaveToCdlqDbAutoSelling] failed to close cursor: 'SaveToCdlqDbAutoSelling' object has no attribute 'cursor'
2026-03-31 21:47:17 [crawlers.pipelines.save_to_cdlq_db_auto_selling] DEBUG: [SaveToCdlqDbAutoSelling] failed to close connection: 'SaveToCdlqDbAutoSelling' object has no attribute 'conn'
2026-03-31 21:47:17 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-31 21:47:17 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-31 21:47:17 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process scrape complete CloudEvent for item 7b23d4662ce811f19724def17024dec4
2026-03-31 21:47:17 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2026-03-31 21:47:17 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2026-03-31 21:47:17 [azure.servicebus._pyamqp.management_link] INFO: Management link receiver state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2026-03-31 21:47:17 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2026-03-31 21:47:17 [azure.servicebus._pyamqp.management_link] INFO: Management link sender state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2026-03-31 21:47:17 [azure.servicebus._pyamqp.session] INFO: Session state changed: <SessionState.MAPPED: 3> -> <SessionState.END_SENT: 4>
2026-03-31 21:47:17 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.OPENED: 9> -> <ConnectionState.CLOSE_SENT: 11>
2026-03-31 21:47:17 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.CLOSE_SENT: 11> -> <ConnectionState.END: 13>
2026-03-31 21:47:17 [azure.servicebus._pyamqp.session] INFO: Session state changed: <SessionState.END_SENT: 4> -> <SessionState.DISCARDING: 6>
2026-03-31 21:47:17 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2026-03-31 21:47:17 [azure.servicebus._pyamqp.management_link] INFO: Management link sender state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2026-03-31 21:47:17 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2026-03-31 21:47:17 [azure.servicebus._pyamqp.management_link] INFO: Management link receiver state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2026-03-31 21:47:17 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2026-03-31 21:47:18 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: [AzureBusPipeline] connection closed
2026-03-31 21:47:18 [auto_selling_leboncoin.fr] INFO: auto_selling_leboncoin.fr Crawl ended with reason finished, scrape types: {<ScrapeType.NEW: 1>: 0, <ScrapeType.NEW_DUPLICATE_ID: 4>: 0, <ScrapeType.PRICE_UPDATE: 2>: 0, <ScrapeType.AUCTION_UPDATE: 3>: 0, <ScrapeType.SKIPPED: 0>: 0}
2026-03-31 21:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-31 21:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_leboncoin.fr%27%20and%20RowKey%20eq%20%273140157498%27 HTTP/1.1" 200 None
2026-03-31 21:47:18 [auto_selling_leboncoin.fr] INFO: Saving data for 3140157498: {'created_time': 1774993638.245216}
2026-03-31 21:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-31 21:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_leboncoin.fr',RowKey='3140157498') HTTP/1.1" 204 0
2026-03-31 21:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-31 21:47:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_leboncoin.fr%27%20and%20last_price_update_time%20lt%201773697638 HTTP/1.1" 200 None
2026-03-31 21:47:18 [scrapy.extensions.feedexport] INFO: Stored jsonlines feed (10 items) in: file:///var/lib/scrapyd/items/sourcing_v2/auto_selling_leboncoin.fr/7b23d4662ce811f19724def17024dec4.jl
2026-03-31 21:47:18 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 704,
'downloader/request_count': 2,
'downloader/request_method_count/GET': 1,
'downloader/request_method_count/POST': 1,
'downloader/response_bytes': 654775,
'downloader/response_count': 2,
'downloader/response_status_count/200': 2,
'elapsed_time_seconds': 11.965636,
'feedexport/success_count/FileFeedStorage': 1,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2026, 3, 31, 21, 47, 18, 428270, tzinfo=datetime.timezone.utc),
'item_scraped_count': 10,
'log_count/DEBUG': 234,
'log_count/ERROR': 1,
'log_count/INFO': 203,
'memusage/max': 149245952,
'memusage/startup': 149245952,
'request_depth_max': 1,
'response_received_count': 2,
'scheduler/dequeued': 2,
'scheduler/dequeued/memory': 2,
'scheduler/enqueued': 2,
'scheduler/enqueued/memory': 2,
'scrape_type/new': 11,
'scrapy-zyte-api/429': 0,
'scrapy-zyte-api/attempts': 2,
'scrapy-zyte-api/error_ratio': 0.0,
'scrapy-zyte-api/errors': 0,
'scrapy-zyte-api/fatal_errors': 0,
'scrapy-zyte-api/mean_connection_seconds': 1.2336223686579615,
'scrapy-zyte-api/mean_response_seconds': 1.4966104915365577,
'scrapy-zyte-api/processed': 2,
'scrapy-zyte-api/request_args/customHttpRequestHeaders': 2,
'scrapy-zyte-api/request_args/experimental.responseCookies': 1,
'scrapy-zyte-api/request_args/httpRequestBody': 1,
'scrapy-zyte-api/request_args/httpRequestMethod': 1,
'scrapy-zyte-api/request_args/httpResponseBody': 2,
'scrapy-zyte-api/request_args/httpResponseHeaders': 2,
'scrapy-zyte-api/request_args/responseCookies': 1,
'scrapy-zyte-api/request_args/url': 2,
'scrapy-zyte-api/status_codes/200': 2,
'scrapy-zyte-api/success': 2,
'scrapy-zyte-api/success_ratio': 1.0,
'scrapy-zyte-api/throttle_ratio': 0.0,
'source/items_encountered': 11,
'source/total_available': 11,
'start_time': datetime.datetime(2026, 3, 31, 21, 47, 6, 462634, tzinfo=datetime.timezone.utc)}
2026-03-31 21:47:18 [scrapy.core.engine] INFO: Spider closed (finished)
2026-03-31 21:47:18 [asyncio] ERROR: Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7fdbbec1af90>
2026-03-31 21:47:18 [asyncio] ERROR: Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7fdbbdba0110>
-
Log
/2/log/utf8/sourcing_v2/auto_selling_leboncoin.fr/7b23d4662ce811f19724def17024dec4/?job_finished=True
-
Source
http://scrapyd-1:6800/logs/sourcing_v2/auto_selling_leboncoin.fr/7b23d4662ce811f19724def17024dec4.log