• LogParser v0.8.2, last updated at 2026-03-31 21:23:52, http://scrapyd-1:6800/logs/sourcing_v2/auto_selling_coches.net.api/7a1a9f282ce811f19724def17024dec4.json

PROJECT (sourcing_v2), SPIDER (auto_selling_coches.net.api)

  • Log analysis
  • Log categorization
  • Progress visualization
  • View log
  • Crawler.stats
  • projectsourcing_v2
    spiderauto_selling_coches.net.api
    job7a1a9f282ce811f19724def17024dec4
    first_log_time2026-03-31 21:21:41
    latest_log_time2026-03-31 21:23:43
    runtime0:02:02
    crawled_pages 101
    scraped_items 96
    shutdown_reasonN/A
    finish_reasonfinished
    log_critical_count0
    log_error_count2
    log_warning_count0
    log_redirect_count0
    log_retry_count0
    log_ignore_count0
    latest_crawl
    latest_scrape
    latest_log
    current_time
    latest_itemN/A
    • WARNING+

    • error_logs
      2 in total

      2026-03-31 21:23:43 [asyncio] ERROR: Unclosed client session
      client_session: <aiohttp.client.ClientSession object at 0x7fc467b1b890>
      2026-03-31 21:23:43 [asyncio] ERROR: Unclosed client session
      client_session: <aiohttp.client.ClientSession object at 0x7fc467b24150>

      INFO

      DEBUG

    • scrapy_version

      2.11.2
    • telnet_console

      127.0.0.1:6024
    • telnet_password

      3524da67aa82d93d
    • latest_crawl

      2026-03-31 21:23:42 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://web.gw.coches.net/details/61999437> (referer: https://www.coches.net/concesionario/autosalonvillanuevaalicante/?pg=4) ['zyte-api']
    • latest_scrape

      2026-03-31 21:23:42 [scrapy.core.scraper] DEBUG: Scraped from <200 https://web.gw.coches.net/details/61999437>
    • latest_stat

      2026-03-31 21:23:41 [scrapy.extensions.logstats] INFO: Crawled 100 pages (at 68 pages/min), scraped 95 items (at 65 items/min)
    • Head

      2026-03-31 21:21:41 [scrapy.utils.log] INFO: Scrapy 2.11.2 started (bot: SourcingV2)
      2026-03-31 21:21:41 [scrapy.utils.log] INFO: Versions: lxml 5.2.2.0, libxml2 2.12.6, cssselect 1.2.0, parsel 1.9.1, w3lib 2.1.2, Twisted 24.3.0, Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0], pyOpenSSL 24.1.0 (OpenSSL 3.2.1 30 Jan 2024), cryptography 42.0.7, Platform Linux-5.15.0-1098-azure-x86_64-with-glibc2.36
      2026-03-31 21:21:41 [auto_selling_coches.net.api] INFO: Starting auto_selling_coches.net.api spider (API-based)
      2026-03-31 21:21:41 [auto_selling_coches.net.api] INFO: Extracted dealer slug: autosalonvillanuevaalicante
      2026-03-31 21:21:41 [auto_selling_coches.net.api] INFO: _job: 7a1a9f282ce811f19724def17024dec4, SCRAPY_JOB: None
      2026-03-31 21:21:41 [scrapy.addons] INFO: Enabled addons:
      []
      2026-03-31 21:21:41 [asyncio] DEBUG: Using selector: EpollSelector
      2026-03-31 21:21:41 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.asyncioreactor.AsyncioSelectorReactor
      2026-03-31 21:21:41 [scrapy.utils.log] DEBUG: Using asyncio event loop: asyncio.unix_events._UnixSelectorEventLoop
      2026-03-31 21:21:41 [scrapy.extensions.telnet] INFO: Telnet Password: 3524da67aa82d93d
      2026-03-31 21:21:41 [scrapy.middleware] INFO: Enabled extensions:
      ['scrapy.extensions.corestats.CoreStats',
       'scrapy.extensions.telnet.TelnetConsole',
       'scrapy.extensions.memusage.MemoryUsage',
       'scrapy.extensions.feedexport.FeedExporter',
       'scrapy.extensions.logstats.LogStats',
       'scrapy.extensions.closespider.CloseSpider']
      2026-03-31 21:21:41 [scrapy.crawler] INFO: Overridden settings:
      {'BOT_NAME': 'SourcingV2',
       'CLOSESPIDER_TIMEOUT': 3600,
       'CONCURRENT_REQUESTS': 5,
       'DOWNLOAD_MAXSIZE': 52428800,
       'DOWNLOAD_WARNSIZE': 10485760,
       'FEED_EXPORT_ENCODING': 'utf-8',
       'LOG_FILE': '/var/log/scrapyd/logs/sourcing_v2/auto_selling_coches.net.api/7a1a9f282ce811f19724def17024dec4.log',
       'LOG_FORMATTER': 'crawlers.log_formatter.SourcingLogFormatter',
       'MEMUSAGE_LIMIT_MB': 2048,
       'MEMUSAGE_WARNING_MB': 1536,
       'NEWSPIDER_MODULE': 'spiders',
       'REQUEST_FINGERPRINTER_CLASS': 'scrapy_zyte_api.ScrapyZyteAPIRequestFingerprinter',
       'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7',
       'SPIDER_MODULES': ['spiders', 'auth_check'],
       'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor',
       'USER_AGENT': ''}
      2026-03-31 21:21:41 [scrapy_zyte_api.handler] INFO: Using a Zyte API key starting with '5857011'
      2026-03-31 21:21:41 [scrapy_zyte_api.handler] INFO: Using a Zyte API key starting with '5857011'
      2026-03-31 21:21:41 [scrapy.middleware] INFO: Enabled downloader middlewares:
      ['scrapy.downloadermiddlewares.offsite.OffsiteMiddleware',
       'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
       'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
       'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
       'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
       'scrapy_zyte_api.ScrapyZyteAPIDownloaderMiddleware',
       'scrapy.downloadermiddlewares.retry.RetryMiddleware',
       'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
       'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
       'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
       'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
       'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
       'scrapy.downloadermiddlewares.stats.DownloaderStats']
      2026-03-31 21:21:41 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Setting up IdGenerationMiddleware for auto_selling
      2026-03-31 21:21:41 [scrapy.middleware] INFO: Enabled spider middlewares:
      ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
       'scrapy_zyte_api.ScrapyZyteAPISpiderMiddleware',
       'scrapy.spidermiddlewares.referer.RefererMiddleware',
       'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
       'scrapy.spidermiddlewares.depth.DepthMiddleware',
       'crawlers.middlewares.photo_download_middleware.PhotoDownloadMiddleware',
       'crawlers.auto_selling.middlewares.id_gen_middleware.IdGenMiddleware']
      2026-03-31 21:21:41 [scrapy.middleware] INFO: Enabled item pipelines:
      ['crawlers.pipelines.post_to_azurebus_auto_selling.AzureBusPipelineAutoSelling',
       'crawlers.pipelines.save_to_cdlq_db_auto_selling.SaveToCdlqDbAutoSelling']
      2026-03-31 21:21:41 [scrapy.core.engine] INFO: Spider opened
      2026-03-31 21:21:41 [azure.identity._credentials.environment] INFO: Incomplete environment configuration for EnvironmentCredential. These variables are set: AZURE_TENANT_ID, AZURE_CLIENT_ID
      2026-03-31 21:21:41 [common.service_bus.service_bus_client_factory] INFO: ServiceBusClient initialized via DefaultAzureCredential for alx-production-bus.servicebus.windows.net
      2026-03-31 21:21:41 [crawlers.pipelines.base_azure_bus_pipeline] INFO: [AzureBusPipeline] Connected to topic auto.sourcing-v2
      2026-03-31 21:21:41 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
      2026-03-31 21:21:41 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6024
      2026-03-31 21:21:41 [scrapy.extensions.memusage] INFO: Peak memory usage is 142MiB
      2026-03-31 21:21:41 [scrapy-playwright] INFO: Starting download handler
      2026-03-31 21:21:41 [scrapy-playwright] INFO: Starting download handler
      2026-03-31 21:21:46 [auto_selling_coches.net.api] INFO: API spider started with URL: https://www.coches.net/concesionario/autosalonvillanuevaalicante, dealer_slug: autosalonvillanuevaalicante
      2026-03-31 21:21:46 [auto_selling_coches.net.api] INFO: Trying fast dealer API: https://web.gw.coches.net/dealers/autosalonvillanuevaalicante
      2026-03-31 21:21:46 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 1st time calling it.
      2026-03-31 21:21:47 [scrapy.core.engine] DEBUG: Crawled (400) <GET https://web.gw.coches.net/dealers/autosalonvillanuevaalicante> (referer: None) ['zyte-api']
      2026-03-31 21:21:47 [auto_selling_coches.net.api] INFO: Search API failed (Ignoring non-200 response), falling back to browserHtml
      2026-03-31 21:21:47 [auto_selling_coches.net.api] INFO: Using browserHtml fallback path (page 1)
      
      2026-03-31 21:21:47 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 1st time calling it.
      2026-03-31 21:22:17 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.coches.net/concesionario/autosalonvillanuevaalicante?pg=1> (referer: None) ['zyte-api']
      2026-03-31 21:22:17 [auto_selling_coches.net.api] INFO: Total items from HTML header: 96
      2026-03-31 21:22:17 [auto_selling_coches.net.api] INFO: Extracting listings from HTML page 1
      2026-03-31 21:22:18 [auto_selling_coches.net.api] INFO: Found 30 listings in HTML (page 1)
      2026-03-31 21:22:18 [auto_selling_coches.net.api] INFO: Found listing with ID: 70282627
      2026-03-31 21:22:18 [azure.identity._credentials.environment] INFO: Incomplete environment configuration for EnvironmentCredential. These variables are set: AZURE_TENANT_ID, AZURE_CLIENT_ID
      2026-03-31 21:22:18 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
      2026-03-31 21:22:18 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 2113
      2026-03-31 21:22:18 [azure.identity._credentials.chained] INFO: DefaultAzureCredential acquired a token from WorkloadIdentityCredential
      2026-03-31 21:22:18 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxsourcingstorageprod.table.core.windows.net:443
      2026-03-31 21:22:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
      2026-03-31 21:22:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_coches.net.api%27%20and%20RowKey%20eq%20%2770282627%27 HTTP/1.1" 200 None
      2026-03-31 21:22:18 [auto_selling_coches.net.api] INFO: Scrape type for 70282627: 1
      2026-03-31 21:22:18 [auto_selling_coches.net.api] INFO: Scraping new listing 70282627
      2026-03-31 21:22:18 [auto_selling_coches.net.api] INFO: Found listing with ID: 70279666
      2026-03-31 21:22:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
      2026-03-31 21:22:18 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_coches.net.api%27%20and%20RowKey%20eq%20%2770279666%27 HTTP/1.1" 200 None
      2026-03-31 21:22:18 [auto_selling_coches.net.api] INFO: Scrape type for 70279666: 1
      2026-03-31 21:22:18 [auto_selling_coches.net.api] INFO: Scraping new listing 70279666
      2026-03-31 21:22:18 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 1st time calling it.
      2026-03-31 21:22:18 [auto_selling_coches.net.api] INFO: Found listing with ID: 70270712
    • Tail

                   'Height': 600,
                   'OriginUrl': 'https://a.ccdn.es/cnet/vehicles/18990867/0cb7f5a2-8e8c-4b3e-a2f0-f78b5caca1ee.jpg',
                   'Width': 800}],
       'price': 159900,
       'price_includes_vat': True,
       'registration_date': '2024-01-01',
       'seats_number': 4,
       'seller_name': None,
       'title': 'PORSCHE Cayenne Coupé Turbo EHybrid',
       'transmission': 'automatic',
       'trim': 'Turbo EHybrid',
       'vin': None,
       'year': 2024}
      2026-03-31 21:23:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
      2026-03-31 21:23:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_coches.net.api%27%20and%20RowKey%20eq%20%2762029097%27 HTTP/1.1" 200 None
      2026-03-31 21:23:41 [auto_selling_coches.net.api] INFO: Saving data for 62029097: {'created_time': 1774992221.575623, 'last_price_update_time': 1774992221.575632}
      2026-03-31 21:23:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
      2026-03-31 21:23:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_coches.net.api',RowKey='62029097') HTTP/1.1" 204 0
      2026-03-31 21:23:41 [scrapy.extensions.logstats] INFO: Crawled 100 pages (at 68 pages/min), scraped 95 items (at 65 items/min)
      2026-03-31 21:23:41 [scrapy.extensions.memusage] INFO: Peak memory usage is 188MiB
      2026-03-31 21:23:42 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://web.gw.coches.net/details/61999437> (referer: https://www.coches.net/concesionario/autosalonvillanuevaalicante/?pg=4) ['zyte-api']
      2026-03-31 21:23:42 [auto_selling_coches.net.api] INFO: Extracted item from API: 61999437 - MERCEDES-BENZ GLE Coupé MercedesAMG GLE 53 4MATIC
      2026-03-31 21:23:42 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: 79ec4f3c-9619-5827-98d2-c608fc51b3aa with identifier: 61999437
      2026-03-31 21:23:42 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 79ec4f3c-9619-5827-98d2-c608fc51b3aa
      2026-03-31 21:23:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
      2026-03-31 21:23:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_coches.net.api%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fa.ccdn.es%252Fcnet%252Fvehicles%252F18964733%252Fa95c81d4-ba86-420d-b6ba-abacd04d9277.jpg%27 HTTP/1.1" 200 None
      2026-03-31 21:23:42 [crawlers.middlewares.photo_download_middleware] INFO: Image https://a.ccdn.es/cnet/vehicles/18964733/a95c81d4-ba86-420d-b6ba-abacd04d9277.jpg already downloaded
      2026-03-31 21:23:42 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: 79ec4f3c-9619-5827-98d2-c608fc51b3aa with identifier 61999437
      2026-03-31 21:23:42 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
      2026-03-31 21:23:42 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
      2026-03-31 21:23:42 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item 79ec4f3c-9619-5827-98d2-c608fc51b3aa
      2026-03-31 21:23:42 [scrapy.core.scraper] DEBUG: Scraped from <200 https://web.gw.coches.net/details/61999437>
      {'auction_title': '',
       'body_type': 'SUV',
       'c_o_2_emission_value': 219,
       'category': 'Coches',
       'color': 'Negro',
       'currency': 'EUR',
       'description': 'MERCEDES-BENZ GLE 53 AMG COUPE 435CV\n'
                      '\n'
                      'Color de carrocería negro profundo\n'
                      'Llantas 21” AMG\n'
                      'Techo panorámico/abatible\n'
                      'Cortinilla superior eléctrica\n'
                      'Faros led inteligentes/automáticos\n'
                      'Pilotos de led\n'
                      'Espejos abatibles automáticos con antideslumbramiento color '
                      'negro\n'
                      'Molduras shadowline color negro\n'
                      'Iluminación ambiental interior\n'
                      'Amg line interior y exterior\n'
                      'Asientos mixtos de cuero eléctricos, calefactables y con '
                      'ajuste lumbar\n'
                      'Volante en cuero negro perforado multifunción con levas \n'
                      'Interior cuero negro\n'
                      'Navegador\n'
                      'Control por voz\n'
                      'Sistema de sonido Burmester\n'
                      'Sistema multimedia digital\n'
                      'Cámaras de aparcamiento con sensores 360 grados\n'
                      'Apple car play\n'
                      'Sistema ACC\n'
                      'Cámara multifuncional\n'
                      'Sistema adaptativo velocidad\n'
                      'Mas extras…..\n'
                      '\n'
                      '-1 año de garantía y transferencia de nombre incluida en el '
                      'precio.\n'
                      '-Consulte nuestras opciones de financiación si lo desea (el '
                      'precio no está sujeto a financiar con nosotros '
                      'directamente).\n'
                      '-Servicio de entrega a domicilio en cualquier punto de la '
                      'Península y Baleares, precio a consultar.\n'
                      '-Solicite prueba de vehículo a domicilio.\n'
                      '-Todos nuestros vehículos disponen de libro de mantenimiento '
                      'y certificación de kilometraje.\n'
                      '-Atendemos con cita previa, contacte con nosotros para '
                      'solicitar una prueba del vehículo.\n'
                      '-Este anuncio no es vinculante, puede estar sujeto a errores, '
                      'se muestra a titulo informativo y no contractual.\n'
                      '-19 años de experiencia nos avalan en el sector del '
                      'automóvil.\n'
                      '-We speak English',
       'doors_number': 5,
       'emission_standard': None,
       'engine': '3.0L',
       'engine_horse_power': 435,
       'expiration_date': None,
       'fuel_type': 'Híbrido',
       'id': '79ec4f3c-9619-5827-98d2-c608fc51b3aa',
       'is_damaged': False,
       'is_operable': True,
       'license_plate': None,
       'link_to_web_offer': 'https://www.coches.net/mercedes-benz-gle-coupe-mercedesamg-gle-53-4matic-5p-electrico-hibrido-2020-en-alicante-61999437-covo.aspx',
       'listing': 'auction',
       'location': 'Alicante',
       'make': 'MERCEDES-BENZ',
       'model': 'GLE Coupé',
       'odometer': 69000,
       'odometer_reading_unit': 'km',
       'origin_country_code': 'ES',
       'original_photo_count': 1,
       'photo_urls': ['https://a.ccdn.es/cnet/vehicles/18964733/a95c81d4-ba86-420d-b6ba-abacd04d9277.jpg'],
       'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_79ec4f3c-9619-5827-98d2-c608fc51b3aa/79ec4f3c-9619-5827-98d2-c608fc51b3aa_image_0.jpg',
                   'Extension': 'jpg',
                   'FileSizeInBytes': 72814,
                   'Height': 567,
                   'OriginUrl': 'https://a.ccdn.es/cnet/vehicles/18964733/a95c81d4-ba86-420d-b6ba-abacd04d9277.jpg',
                   'Width': 757}],
       'price': 78900,
       'price_includes_vat': True,
       'registration_date': '2021-01-01',
       'seats_number': 5,
       'seller_name': None,
       'title': 'MERCEDES-BENZ GLE Coupé MercedesAMG GLE 53 4MATIC',
       'transmission': 'automatic',
       'trim': 'MercedesAMG GLE 53 4MATIC',
       'vin': None,
       'year': 2021}
      2026-03-31 21:23:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
      2026-03-31 21:23:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_coches.net.api%27%20and%20RowKey%20eq%20%2761999437%27 HTTP/1.1" 200 None
      2026-03-31 21:23:42 [auto_selling_coches.net.api] INFO: Saving data for 61999437: {'created_time': 1774992222.263172, 'last_price_update_time': 1774992222.263183}
      2026-03-31 21:23:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
      2026-03-31 21:23:42 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_coches.net.api',RowKey='61999437') HTTP/1.1" 204 0
      2026-03-31 21:23:42 [scrapy.core.engine] INFO: Closing spider (finished)
      2026-03-31 21:23:42 [crawlers.pipelines.save_to_cdlq_db_auto_selling] DEBUG: [SaveToCdlqDbAutoSelling] failed to close cursor: 'SaveToCdlqDbAutoSelling' object has no attribute 'cursor'
      2026-03-31 21:23:42 [crawlers.pipelines.save_to_cdlq_db_auto_selling] DEBUG: [SaveToCdlqDbAutoSelling] failed to close connection: 'SaveToCdlqDbAutoSelling' object has no attribute 'conn'
      2026-03-31 21:23:42 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
      2026-03-31 21:23:42 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
      2026-03-31 21:23:42 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process scrape complete CloudEvent for item 7a1a9f282ce811f19724def17024dec4
      2026-03-31 21:23:42 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
      2026-03-31 21:23:42 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
      2026-03-31 21:23:42 [azure.servicebus._pyamqp.management_link] INFO: Management link receiver state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
      2026-03-31 21:23:42 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
      2026-03-31 21:23:42 [azure.servicebus._pyamqp.management_link] INFO: Management link sender state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
      2026-03-31 21:23:42 [azure.servicebus._pyamqp.session] INFO: Session state changed: <SessionState.MAPPED: 3> -> <SessionState.END_SENT: 4>
      2026-03-31 21:23:42 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.OPENED: 9> -> <ConnectionState.CLOSE_SENT: 11>
      2026-03-31 21:23:42 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.CLOSE_SENT: 11> -> <ConnectionState.END: 13>
      2026-03-31 21:23:42 [azure.servicebus._pyamqp.session] INFO: Session state changed: <SessionState.END_SENT: 4> -> <SessionState.DISCARDING: 6>
      2026-03-31 21:23:42 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
      2026-03-31 21:23:42 [azure.servicebus._pyamqp.management_link] INFO: Management link sender state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
      2026-03-31 21:23:42 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
      2026-03-31 21:23:42 [azure.servicebus._pyamqp.management_link] INFO: Management link receiver state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
      2026-03-31 21:23:42 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
      2026-03-31 21:23:43 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: [AzureBusPipeline] connection closed
      2026-03-31 21:23:43 [auto_selling_coches.net.api] INFO: auto_selling_coches.net.api Crawl ended with reason finished, scrape types: {<ScrapeType.NEW: 1>: 6, <ScrapeType.NEW_DUPLICATE_ID: 4>: 0, <ScrapeType.PRICE_UPDATE: 2>: 0, <ScrapeType.AUCTION_UPDATE: 3>: 0, <ScrapeType.SKIPPED: 0>: 0}
      2026-03-31 21:23:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
      2026-03-31 21:23:43 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_coches.net.api%27%20and%20last_price_update_time%20lt%201773696223 HTTP/1.1" 200 None
      2026-03-31 21:23:43 [scrapy.extensions.feedexport] INFO: Stored jsonlines feed (96 items) in: file:///var/lib/scrapyd/items/sourcing_v2/auto_selling_coches.net.api/7a1a9f282ce811f19724def17024dec4.jl
      2026-03-31 21:23:43 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
      {'downloader/request_bytes': 166800,
       'downloader/request_count': 101,
       'downloader/request_method_count/GET': 101,
       'downloader/response_bytes': 9029324,
       'downloader/response_count': 101,
       'downloader/response_status_count/200': 100,
       'downloader/response_status_count/400': 1,
       'elapsed_time_seconds': 121.959173,
       'feedexport/success_count/FileFeedStorage': 1,
       'finish_reason': 'finished',
       'finish_time': datetime.datetime(2026, 3, 31, 21, 23, 43, 442701, tzinfo=datetime.timezone.utc),
       'item_scraped_count': 96,
       'log_count/DEBUG': 1376,
       'log_count/INFO': 945,
       'memusage/max': 198037504,
       'memusage/startup': 149753856,
       'request_depth_max': 5,
       'response_received_count': 101,
       'scheduler/dequeued': 101,
       'scheduler/dequeued/memory': 101,
       'scheduler/enqueued': 101,
       'scheduler/enqueued/memory': 101,
       'scrape_type/new': 96,
       'scrapy-zyte-api/429': 0,
       'scrapy-zyte-api/attempts': 101,
       'scrapy-zyte-api/error_ratio': 0.0,
       'scrapy-zyte-api/errors': 0,
       'scrapy-zyte-api/fatal_errors': 0,
       'scrapy-zyte-api/mean_connection_seconds': 2.2419747672237387,
       'scrapy-zyte-api/mean_response_seconds': 2.308148939268533,
       'scrapy-zyte-api/processed': 101,
       'scrapy-zyte-api/request_args/actions': 4,
       'scrapy-zyte-api/request_args/browserHtml': 4,
       'scrapy-zyte-api/request_args/customHttpRequestHeaders': 97,
       'scrapy-zyte-api/request_args/experimental.requestCookies': 100,
       'scrapy-zyte-api/request_args/experimental.responseCookies': 101,
       'scrapy-zyte-api/request_args/httpResponseBody': 97,
       'scrapy-zyte-api/request_args/httpResponseHeaders': 97,
       'scrapy-zyte-api/request_args/requestHeaders': 3,
       'scrapy-zyte-api/request_args/url': 101,
       'scrapy-zyte-api/status_codes/200': 101,
       'scrapy-zyte-api/success': 101,
       'scrapy-zyte-api/success_ratio': 1.0,
       'scrapy-zyte-api/throttle_ratio': 0.0,
       'start_time': datetime.datetime(2026, 3, 31, 21, 21, 41, 483528, tzinfo=datetime.timezone.utc)}
      2026-03-31 21:23:43 [scrapy.core.engine] INFO: Spider closed (finished)
      2026-03-31 21:23:43 [asyncio] ERROR: Unclosed client session
      client_session: <aiohttp.client.ClientSession object at 0x7fc467b1b890>
      2026-03-31 21:23:43 [asyncio] ERROR: Unclosed client session
      client_session: <aiohttp.client.ClientSession object at 0x7fc467b24150>
      
    • Log

      /2/log/utf8/sourcing_v2/auto_selling_coches.net.api/7a1a9f282ce811f19724def17024dec4/?job_finished=True

    • Source

      http://scrapyd-1:6800/logs/sourcing_v2/auto_selling_coches.net.api/7a1a9f282ce811f19724def17024dec4.log

  • sourcelog
    last_update_time2026-03-31 21:23:43
    last_update_timestamp1774992223
    downloader/request_bytes166800
    downloader/request_count101
    downloader/request_method_count/GET101
    downloader/response_bytes9029324
    downloader/response_count101
    downloader/response_status_count/200100
    downloader/response_status_count/4001
    elapsed_time_seconds121.959173
    feedexport/success_count/FileFeedStorage1
    finish_reasonfinished
    finish_timedatetime.datetime(2026, 3, 31, 21, 23, 43, 442701, tzinfo=datetime.timezone.utc)
    item_scraped_count96
    log_count/DEBUG1376
    log_count/INFO945
    memusage/max198037504
    memusage/startup149753856
    request_depth_max5
    response_received_count101
    scheduler/dequeued101
    scheduler/dequeued/memory101
    scheduler/enqueued101
    scheduler/enqueued/memory101
    scrape_type/new96
    scrapy-zyte-api/4290
    scrapy-zyte-api/attempts101
    scrapy-zyte-api/error_ratio0.0
    scrapy-zyte-api/errors0
    scrapy-zyte-api/fatal_errors0
    scrapy-zyte-api/mean_connection_seconds2.2419747672237387
    scrapy-zyte-api/mean_response_seconds2.308148939268533
    scrapy-zyte-api/processed101
    scrapy-zyte-api/request_args/actions4
    scrapy-zyte-api/request_args/browserHtml4
    scrapy-zyte-api/request_args/customHttpRequestHeaders97
    scrapy-zyte-api/request_args/experimental.requestCookies100
    scrapy-zyte-api/request_args/experimental.responseCookies101
    scrapy-zyte-api/request_args/httpResponseBody97
    scrapy-zyte-api/request_args/httpResponseHeaders97
    scrapy-zyte-api/request_args/requestHeaders3
    scrapy-zyte-api/request_args/url101
    scrapy-zyte-api/status_codes/200101
    scrapy-zyte-api/success101
    scrapy-zyte-api/success_ratio1.0
    scrapy-zyte-api/throttle_ratio0.0
    start_timedatetime.datetime(2026, 3, 31, 21, 21, 41, 483528, tzinfo=datetime.timezone.utc)