Head
2026-03-25 15:27:34 [scrapy.utils.log] INFO: Scrapy 2.11.2 started (bot: SourcingV2)
2026-03-25 15:27:34 [scrapy.utils.log] INFO: Versions: lxml 5.2.2.0, libxml2 2.12.6, cssselect 1.2.0, parsel 1.9.1, w3lib 2.1.2, Twisted 24.3.0, Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0], pyOpenSSL 24.1.0 (OpenSSL 3.2.1 30 Jan 2024), cryptography 42.0.7, Platform Linux-5.15.0-1098-azure-x86_64-with-glibc2.36
2026-03-25 15:27:34 [auto_selling_leboncoin.fr] INFO: Starting auto_selling_leboncoin.fr spider
2026-03-25 15:27:34 [auto_selling_leboncoin.fr] INFO: Quick mode enabled: limiting photo downloads to 1 per item
2026-03-25 15:27:34 [auto_selling_leboncoin.fr] INFO: _job: 23cefbba285f11f18f76c264925fbcfa, SCRAPY_JOB: None
2026-03-25 15:27:34 [scrapy.addons] INFO: Enabled addons:
[]
2026-03-25 15:27:34 [asyncio] DEBUG: Using selector: EpollSelector
2026-03-25 15:27:34 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.asyncioreactor.AsyncioSelectorReactor
2026-03-25 15:27:34 [scrapy.utils.log] DEBUG: Using asyncio event loop: asyncio.unix_events._UnixSelectorEventLoop
2026-03-25 15:27:34 [scrapy.extensions.telnet] INFO: Telnet Password: 2c6dc7eee23f2935
2026-03-25 15:27:34 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
'scrapy.extensions.telnet.TelnetConsole',
'scrapy.extensions.memusage.MemoryUsage',
'scrapy.extensions.feedexport.FeedExporter',
'scrapy.extensions.logstats.LogStats',
'scrapy.extensions.closespider.CloseSpider']
2026-03-25 15:27:34 [scrapy.crawler] INFO: Overridden settings:
{'BOT_NAME': 'SourcingV2',
'CLOSESPIDER_TIMEOUT': 3600,
'CONCURRENT_REQUESTS': 5,
'DOWNLOAD_MAXSIZE': 52428800,
'DOWNLOAD_WARNSIZE': 10485760,
'FEED_EXPORT_ENCODING': 'utf-8',
'LOG_FILE': '/var/log/scrapyd/logs/sourcing_v2/auto_selling_leboncoin.fr/23cefbba285f11f18f76c264925fbcfa.log',
'LOG_FORMATTER': 'crawlers.log_formatter.SourcingLogFormatter',
'MEMUSAGE_LIMIT_MB': 2048,
'MEMUSAGE_WARNING_MB': 1536,
'NEWSPIDER_MODULE': 'spiders',
'REQUEST_FINGERPRINTER_CLASS': 'scrapy_zyte_api.ScrapyZyteAPIRequestFingerprinter',
'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7',
'SPIDER_MODULES': ['spiders', 'auth_check'],
'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor',
'USER_AGENT': ''}
2026-03-25 15:27:34 [scrapy_zyte_api.handler] INFO: Using a Zyte API key starting with '5857011'
2026-03-25 15:27:34 [scrapy_zyte_api.handler] INFO: Using a Zyte API key starting with '5857011'
2026-03-25 15:27:34 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.offsite.OffsiteMiddleware',
'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
'scrapy_zyte_api.ScrapyZyteAPIDownloaderMiddleware',
'scrapy.downloadermiddlewares.retry.RetryMiddleware',
'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
'scrapy.downloadermiddlewares.stats.DownloaderStats']
2026-03-25 15:27:34 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Setting up IdGenerationMiddleware for auto_selling
2026-03-25 15:27:34 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
'scrapy_zyte_api.ScrapyZyteAPISpiderMiddleware',
'scrapy.spidermiddlewares.referer.RefererMiddleware',
'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
'scrapy.spidermiddlewares.depth.DepthMiddleware',
'crawlers.middlewares.photo_download_middleware.PhotoDownloadMiddleware',
'crawlers.auto_selling.middlewares.id_gen_middleware.IdGenMiddleware']
2026-03-25 15:27:34 [scrapy.middleware] INFO: Enabled item pipelines:
['crawlers.pipelines.post_to_azurebus_auto_selling.AzureBusPipelineAutoSelling',
'crawlers.pipelines.save_to_cdlq_db_auto_selling.SaveToCdlqDbAutoSelling']
2026-03-25 15:27:34 [scrapy.core.engine] INFO: Spider opened
2026-03-25 15:27:34 [azure.identity._credentials.environment] INFO: Incomplete environment configuration for EnvironmentCredential. These variables are set: AZURE_TENANT_ID, AZURE_CLIENT_ID
2026-03-25 15:27:34 [common.service_bus.service_bus_client_factory] INFO: ServiceBusClient initialized via DefaultAzureCredential for alx-production-bus.servicebus.windows.net
2026-03-25 15:27:34 [crawlers.pipelines.base_azure_bus_pipeline] INFO: [AzureBusPipeline] Connected to topic auto.sourcing-v2-quickmode
2026-03-25 15:27:34 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-03-25 15:27:34 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2026-03-25 15:27:34 [scrapy.extensions.memusage] INFO: Peak memory usage is 140MiB
2026-03-25 15:27:34 [scrapy-playwright] INFO: Starting download handler
2026-03-25 15:27:34 [scrapy-playwright] INFO: Starting download handler
2026-03-25 15:27:39 [auto_selling_leboncoin.fr] INFO: Scraper started with URL: https://www.leboncoin.fr/boutique/1417126/tb-auto.htm and multipost_profile_id: 5ea80d2c-ff9f-4039-95a7-5afb4e941fad and job_id: 23cefbba285f11f18f76c264925fbcfa
2026-03-25 15:27:39 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 1st time calling it.
2026-03-25 15:27:52 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.leboncoin.fr/boutique/1417126/tb-auto.htm> (referer: https://www.leboncoin.fr/boutique/1417126/tb-auto.htm) ['zyte-api']
2026-03-25 15:27:52 [auto_selling_leboncoin.fr] INFO: Captured Zyte cookies: [{'name': '__Secure-Install', 'value': '7dd97fcc-5791-4e9b-b302-8fe9c9a9a51a', 'domain': '.leboncoin.fr', 'path': '/', 'expires': 253402300798, 'httpOnly': False, 'secure': True}, {'name': 'cnfdVisitorId', 'value': '8e979e12-8969-4eb7-a949-aa955649dea4', 'domain': '.leboncoin.fr', 'path': '/', 'expires': 1932132471, 'httpOnly': False, 'secure': True}, {'name': 'datadome', 'value': '1MEjGeA3Br18137HSaVkcf67cYTzUDmLrs3Br5aysZKi~46sV0neuaierI4G1UKpKuqNlTbrxgeLJr38mTJVs_1Bc4jx2yc4HDRZmzvG62ZQlSLPc0~c3vPlM1tVUD~c', 'domain': '.leboncoin.fr', 'path': '/', 'expires': 1805988471, 'httpOnly': False, 'secure': True}]
2026-03-25 15:27:52 [auto_selling_leboncoin.fr] INFO: Scrapy-formatted cookies: [{'name': '__Secure-Install', 'value': '7dd97fcc-5791-4e9b-b302-8fe9c9a9a51a', 'domain': '.leboncoin.fr', 'path': '/'}, {'name': 'cnfdVisitorId', 'value': '8e979e12-8969-4eb7-a949-aa955649dea4', 'domain': '.leboncoin.fr', 'path': '/'}, {'name': 'datadome', 'value': '1MEjGeA3Br18137HSaVkcf67cYTzUDmLrs3Br5aysZKi~46sV0neuaierI4G1UKpKuqNlTbrxgeLJr38mTJVs_1Bc4jx2yc4HDRZmzvG62ZQlSLPc0~c3vPlM1tVUD~c', 'domain': '.leboncoin.fr', 'path': '/'}]
2026-03-25 15:27:52 [auto_selling_leboncoin.fr] INFO: Response headers after parse_dealer_home_page: {b'X-Amz-Cf-Pop': [b'CDG50-P2'], b'Date': [b'Wed, 25 Mar 2026 15:27:51 GMT'], b'Vary': [b'Accept-Encoding'], b'Accept-Ch': [b'Sec-CH-UA,Sec-CH-UA-Mobile,Sec-CH-UA-Platform,Sec-CH-UA-Arch,Sec-CH-UA-Full-Version-List,Sec-CH-UA-Model,Sec-CH-Device-Memory'], b'Strict-Transport-Security': [b'max-age=15768000'], b'Via': [b'1.1 7aef920ed20c713960127526fa3a88f4.cloudfront.net (CloudFront)'], b'Set-Cookie': [b'datadome=1MEjGeA3Br18137HSaVkcf67cYTzUDmLrs3Br5aysZKi~46sV0neuaierI4G1UKpKuqNlTbrxgeLJr38mTJVs_1Bc4jx2yc4HDRZmzvG62ZQlSLPc0~c3vPlM1tVUD~c; Max-Age=31536000; Domain=.leboncoin.fr; Path=/; Secure; SameSite=Lax'], b'Content-Security-Policy': [b'frame-ancestors *.leboncoin.fr *.leboncoin.io *.leboncoin.ci; report-uri https://api.leboncoin.fr/api/csp-report/v1/report/;'], b'Referrer-Policy': [b'no-referrer-when-downgrade'], b'Content-Security-Policy-Report-Only': [b'object-src *.leboncoin.fr *.leboncoin.io *.leboncoin.ci; frame-ancestors *.leboncoin.fr *.leboncoin.io *.leboncoin.ci; report-uri https://api.leboncoin.fr/api/csp-report/v1/report/;'], b'Etag': [b'W/"11vrzaxukb9cbx0"'], b'Content-Type': [b'text/html; charset=utf-8'], b'X-Cache': [b'Miss from cloudfront'], b'X-Datadome': [b'protected'], b'Cache-Control': [b'private, no-cache, no-store, max-age=0, must-revalidate'], b'X-Amz-Cf-Id': [b'ZrlHxwWHmcf1PhVAiok7FQ6Rw5TYggJcqmIpDESySLvcHoE0Pxisow==']}
2026-03-25 15:27:52 [auto_selling_leboncoin.fr] INFO: defaultSearchOptions: {'limit': 10, 'filters': {'owner': {'user_id': '90cb00e2-0aff-4006-bfe0-5d6467c364ce'}, 'enums': {'ad_type': ['offer']}, 'category': {'id': '1'}}}
2026-03-25 15:27:52 [auto_selling_leboncoin.fr] INFO: cookies: [{'name': '__Secure-Install', 'value': '7dd97fcc-5791-4e9b-b302-8fe9c9a9a51a', 'domain': '.leboncoin.fr', 'path': '/'}, {'name': 'cnfdVisitorId', 'value': '8e979e12-8969-4eb7-a949-aa955649dea4', 'domain': '.leboncoin.fr', 'path': '/'}, {'name': 'datadome', 'value': '1MEjGeA3Br18137HSaVkcf67cYTzUDmLrs3Br5aysZKi~46sV0neuaierI4G1UKpKuqNlTbrxgeLJr38mTJVs_1Bc4jx2yc4HDRZmzvG62ZQlSLPc0~c3vPlM1tVUD~c', 'domain': '.leboncoin.fr', 'path': '/'}]
2026-03-25 15:27:52 [auto_selling_leboncoin.fr] INFO: Payload for get listings: {'limit': 200, 'filters': {'owner': {'user_id': '90cb00e2-0aff-4006-bfe0-5d6467c364ce'}, 'enums': {'ad_type': ['offer']}, 'category': {'id': '1'}}, 'offset': 0}
2026-03-25 15:27:52 [auto_selling_leboncoin.fr] INFO: Request: curl -X POST https://api.leboncoin.fr/finder/search --data-raw '{"filters": {"category": {"id": "1"}, "enums": {"ad_type": ["offer"]}, "owner": {"user_id": "90cb00e2-0aff-4006-bfe0-5d6467c364ce"}}, "limit": 200, "offset": 0}' -H 'Accept: */*' -H 'Accept-Language: en-GB,en-US;q=0.9,en;q=0.8' -H 'Content-Type: application/json' -H 'Origin: https://www.leboncoin.fr' --cookie 'name=__Secure-Install; name=cnfdVisitorId; name=datadome'
2026-03-25 15:27:52 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 1st time calling it.
2026-03-25 15:28:04 [scrapy.core.engine] DEBUG: Crawled (200) <POST https://api.leboncoin.fr/finder/search> (referer: https://www.leboncoin.fr/boutique/1417126/tb-auto.htm) ['zyte-api']
2026-03-25 15:28:05 [auto_selling_leboncoin.fr] INFO: Found 38 listings on page 1
2026-03-25 15:28:05 [auto_selling_leboncoin.fr] INFO: Found listing with ID: 3131591306
2026-03-25 15:28:05 [azure.identity._credentials.environment] INFO: Incomplete environment configuration for EnvironmentCredential. These variables are set: AZURE_TENANT_ID, AZURE_CLIENT_ID
2026-03-25 15:28:05 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
2026-03-25 15:28:05 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 2113
2026-03-25 15:28:05 [azure.identity._credentials.chained] INFO: DefaultAzureCredential acquired a token from WorkloadIdentityCredential
2026-03-25 15:28:05 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxsourcingstorageprod.table.core.windows.net:443
2026-03-25 15:28:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 15:28:05 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_leboncoin.fr%27%20and%20RowKey%20eq%20%273131591306%27 HTTP/1.1" 200 None
2026-03-25 15:28:05 [auto_selling_leboncoin.fr] INFO: Scrape type for 3131591306: 1
2026-03-25 15:28:05 [auto_selling_leboncoin.fr] INFO: Scraping new listing 3131591306
2026-03-25 15:28:05 [auto_selling_leboncoin.fr] INFO: Extracted item: {'auction_title': '',
'c_o_2_emission_value': None,
'category': None,
'color': 'Gris',
'currency': 'EUR',
Tail
'price_includes_vat': False,
'registration_date': '2012-11-01',
'seats_number': 4,
'seller_name': None,
'title': 'PEUGEOT 107 1.0 12V 68cv URBAN 3P',
'transmission': 'manual',
'trim': '107 1.0 12v Urban Euro5 3p',
'vin': None,
'year': 2012}
2026-03-25 15:28:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 15:28:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_leboncoin.fr%27%20and%20RowKey%20eq%20%273134919182%27 HTTP/1.1" 200 None
2026-03-25 15:28:12 [auto_selling_leboncoin.fr] INFO: Saving data for 3134919182: {'created_time': 1774452492.784139, 'last_price_update_time': 1774452492.784149}
2026-03-25 15:28:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 15:28:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_leboncoin.fr',RowKey='3134919182') HTTP/1.1" 204 0
2026-03-25 15:28:12 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: afe8c3d3-6469-5045-bdb8-568a4cf33349 with identifier 3128257544
2026-03-25 15:28:12 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 15:28:12 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 15:28:12 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item afe8c3d3-6469-5045-bdb8-568a4cf33349
2026-03-25 15:28:12 [scrapy.core.scraper] DEBUG: Scraped from <200 https://img.leboncoin.fr/api/v1/lbcpb1/images/7e/52/ba/7e52ba82ed7ce6f8f89c5943c4e861bb5637dfc9.jpg?rule=ad-large>
{'auction_title': '',
'c_o_2_emission_value': None,
'category': None,
'color': 'Blanc',
'currency': 'EUR',
'description': '30/06/2006\n'
'double des clefs\n'
'véhicule Français \n'
'06cv\n'
'\n'
'Extérieur et Châssis\n'
'2 roues motrices\n'
'Clignotants blancs\n'
'Lunette arrière dégivrante\n'
'Peinture intégrale\n'
'Peinture métallisée\n'
'Rétroviseurs rabattables\n'
'\n'
'Intérieur\n'
'Banquette 1/3 - 2/3\n'
'Banquette rabattable\n'
'Climatisation\n'
'Couvre bagages\n'
'Direction assistée\n'
'Éclairage intérieur temporisé\n'
'Fermeture électrique\n'
'Siège conducteur réglable hauteur\n'
'Vitres électriques\n'
'Volant 3 branches\n'
'Volant et pommeau cuir\n'
'APPLE CAR PLAY\n'
'Android Auto\n'
'\n'
'Sécurité\n'
'3 ceintures ar. 3 points\n'
'3e feu stop\n'
'ABS\n'
'Airbags\n'
'ESP\n'
'Prétensionneurs de ceinture\n'
'\n'
'Antivol\n'
'Anti démarrage\n'
'\n'
'Autre\n'
'Kit de réparation crevaison\n'
'Non fumeur\n'
'\n'
'VÉHICULE RÉVISÉ ET GARANTIE\n'
'REPRISE ET FINANCEMENT POSSIBLE\n'
'\n'
'TB AUTOMOBILE\n'
"132 CHEMIN DE L'HOMME MORT\n"
'30 900 NÎMES ( Face à Peugeot, collé a Ford)\n'
'\n'
'RETROUVEZ TOUS NOS VÉHICULES\n'
'EN TAPANT TB AUTOMOBILE DANS\n'
'VOTRE MOTEUR DE RECHERCHE',
'doors_number': 5,
'emission_standard': None,
'engine': None,
'engine_horse_power': 90,
'expiration_date': None,
'fuel_type': 'Essence',
'id': 'afe8c3d3-6469-5045-bdb8-568a4cf33349',
'is_damaged': False,
'is_operable': True,
'license_plate': None,
'link_to_web_offer': 'https://www.leboncoin.fr/ad/voitures/3128257544',
'listing': 'auction',
'make': 'PEUGEOT',
'model': '207',
'odometer': 141000,
'odometer_reading_unit': 'km',
'origin_country_code': 'FR',
'original_photo_count': 5,
'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_afe8c3d3-6469-5045-bdb8-568a4cf33349/afe8c3d3-6469-5045-bdb8-568a4cf33349_image_0.jpg',
'Extension': 'jpg',
'FileSizeInBytes': 167179,
'Height': 800,
'OriginUrl': 'https://img.leboncoin.fr/api/v1/lbcpb1/images/7e/52/ba/7e52ba82ed7ce6f8f89c5943c4e861bb5637dfc9.jpg?rule=ad-large',
'Width': 1200}],
'price': 4490,
'price_includes_vat': False,
'registration_date': '2006-06-01',
'seats_number': 5,
'seller_name': None,
'title': 'PEUGEOT 207 1.4 16V 90cv URBAN 3P',
'transmission': 'manual',
'trim': None,
'vin': None,
'year': 2006}
2026-03-25 15:28:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 15:28:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_leboncoin.fr%27%20and%20RowKey%20eq%20%273128257544%27 HTTP/1.1" 200 None
2026-03-25 15:28:12 [auto_selling_leboncoin.fr] INFO: Saving data for 3128257544: {'created_time': 1774452492.842562, 'last_price_update_time': 1774452492.842572}
2026-03-25 15:28:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 15:28:12 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_leboncoin.fr',RowKey='3128257544') HTTP/1.1" 204 0
2026-03-25 15:28:12 [scrapy.core.engine] INFO: Closing spider (finished)
2026-03-25 15:28:12 [crawlers.pipelines.save_to_cdlq_db_auto_selling] DEBUG: [SaveToCdlqDbAutoSelling] failed to close cursor: 'SaveToCdlqDbAutoSelling' object has no attribute 'cursor'
2026-03-25 15:28:12 [crawlers.pipelines.save_to_cdlq_db_auto_selling] DEBUG: [SaveToCdlqDbAutoSelling] failed to close connection: 'SaveToCdlqDbAutoSelling' object has no attribute 'conn'
2026-03-25 15:28:12 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 15:28:12 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
2026-03-25 15:28:12 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process scrape complete CloudEvent for item 23cefbba285f11f18f76c264925fbcfa
2026-03-25 15:28:12 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2026-03-25 15:28:12 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2026-03-25 15:28:12 [azure.servicebus._pyamqp.management_link] INFO: Management link receiver state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2026-03-25 15:28:12 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2026-03-25 15:28:12 [azure.servicebus._pyamqp.management_link] INFO: Management link sender state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
2026-03-25 15:28:12 [azure.servicebus._pyamqp.session] INFO: Session state changed: <SessionState.MAPPED: 3> -> <SessionState.END_SENT: 4>
2026-03-25 15:28:12 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.OPENED: 9> -> <ConnectionState.CLOSE_SENT: 11>
2026-03-25 15:28:12 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.CLOSE_SENT: 11> -> <ConnectionState.END: 13>
2026-03-25 15:28:12 [azure.servicebus._pyamqp.session] INFO: Session state changed: <SessionState.END_SENT: 4> -> <SessionState.DISCARDING: 6>
2026-03-25 15:28:12 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2026-03-25 15:28:12 [azure.servicebus._pyamqp.management_link] INFO: Management link sender state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2026-03-25 15:28:12 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2026-03-25 15:28:12 [azure.servicebus._pyamqp.management_link] INFO: Management link receiver state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2026-03-25 15:28:12 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
2026-03-25 15:28:13 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: [AzureBusPipeline] connection closed
2026-03-25 15:28:13 [auto_selling_leboncoin.fr] INFO: auto_selling_leboncoin.fr Crawl ended with reason finished, scrape types: {<ScrapeType.NEW: 1>: 7, <ScrapeType.NEW_DUPLICATE_ID: 4>: 0, <ScrapeType.PRICE_UPDATE: 2>: 0, <ScrapeType.AUCTION_UPDATE: 3>: 0, <ScrapeType.SKIPPED: 0>: 0}
2026-03-25 15:28:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 15:28:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_leboncoin.fr%27%20and%20RowKey%20eq%20%273154033844%27 HTTP/1.1" 200 None
2026-03-25 15:28:14 [auto_selling_leboncoin.fr] INFO: Saving data for 3154033844: {'created_time': 1774452494.182661}
2026-03-25 15:28:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 15:28:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_leboncoin.fr',RowKey='3154033844') HTTP/1.1" 204 0
2026-03-25 15:28:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
2026-03-25 15:28:14 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_leboncoin.fr%27%20and%20last_price_update_time%20lt%201773156494 HTTP/1.1" 200 None
2026-03-25 15:28:14 [scrapy.extensions.feedexport] INFO: Stored jsonlines feed (37 items) in: file:///var/lib/scrapyd/items/sourcing_v2/auto_selling_leboncoin.fr/23cefbba285f11f18f76c264925fbcfa.jl
2026-03-25 15:28:14 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 23385,
'downloader/request_count': 39,
'downloader/request_method_count/GET': 38,
'downloader/request_method_count/POST': 1,
'downloader/response_bytes': 7830642,
'downloader/response_count': 39,
'downloader/response_status_count/200': 39,
'elapsed_time_seconds': 39.512155,
'feedexport/success_count/FileFeedStorage': 1,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2026, 3, 25, 15, 28, 14, 340232, tzinfo=datetime.timezone.utc),
'item_scraped_count': 37,
'log_count/DEBUG': 664,
'log_count/ERROR': 1,
'log_count/INFO': 666,
'memusage/max': 147279872,
'memusage/startup': 147279872,
'photo_download_count': 37,
'request_depth_max': 2,
'response_received_count': 2,
'scheduler/dequeued': 39,
'scheduler/dequeued/memory': 39,
'scheduler/enqueued': 39,
'scheduler/enqueued/memory': 39,
'scrape_type/new': 38,
'scrapy-zyte-api/429': 0,
'scrapy-zyte-api/attempts': 2,
'scrapy-zyte-api/error_ratio': 0.0,
'scrapy-zyte-api/errors': 0,
'scrapy-zyte-api/fatal_errors': 0,
'scrapy-zyte-api/mean_connection_seconds': 12.176359602250159,
'scrapy-zyte-api/mean_response_seconds': 12.502414725488052,
'scrapy-zyte-api/processed': 2,
'scrapy-zyte-api/request_args/customHttpRequestHeaders': 2,
'scrapy-zyte-api/request_args/experimental.responseCookies': 1,
'scrapy-zyte-api/request_args/httpRequestBody': 1,
'scrapy-zyte-api/request_args/httpRequestMethod': 1,
'scrapy-zyte-api/request_args/httpResponseBody': 2,
'scrapy-zyte-api/request_args/httpResponseHeaders': 2,
'scrapy-zyte-api/request_args/responseCookies': 1,
'scrapy-zyte-api/request_args/url': 2,
'scrapy-zyte-api/status_codes/200': 2,
'scrapy-zyte-api/success': 2,
'scrapy-zyte-api/success_ratio': 1.0,
'scrapy-zyte-api/throttle_ratio': 0.0,
'source/items_encountered': 38,
'source/total_available': 38,
'start_time': datetime.datetime(2026, 3, 25, 15, 27, 34, 828077, tzinfo=datetime.timezone.utc)}
2026-03-25 15:28:14 [scrapy.core.engine] INFO: Spider closed (finished)
2026-03-25 15:28:14 [asyncio] ERROR: Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7fa5fcf8f150>
2026-03-25 15:28:14 [asyncio] ERROR: Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7fa5fcf8f990>