Desktop version Jobs
  • Analysis
  • Categories
  • Charts
  • Logs
  • Crawler.stats
  • projectsourcing_v2
    spiderauto_selling_coches.net.api
    job4a3f36662c1f11f18e133aacc95538c6
    first_log_time2026-03-30 14:58:37
    latest_log_time2026-03-30 15:03:41
    runtime0:05:04
    crawled_pages 101
    scraped_items 96
    shutdown_reasonN/A
    finish_reasonfinished
    log_critical_count0
    log_error_count2
    log_warning_count0
    log_redirect_count0
    log_retry_count0
    log_ignore_count0
    latest_crawl
    latest_scrape
    latest_log
    current_time
    latest_itemN/A
    • WARNING+

    • error_logs
      2 in total

      2026-03-30 15:03:41 [asyncio] ERROR: Unclosed client session
      client_session: <aiohttp.client.ClientSession object at 0x7f1054122f50>
      2026-03-30 15:03:41 [asyncio] ERROR: Unclosed client session
      client_session: <aiohttp.client.ClientSession object at 0x7f1054123790>

      INFO

      DEBUG

    • scrapy_version

      2.11.2
    • telnet_console

      127.0.0.1:6024
    • telnet_password

      4d9b6cfeb54682b6
    • latest_crawl

      2026-03-30 15:03:40 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://web.gw.coches.net/details/60727750> (referer: https://www.coches.net/concesionario/autosalonvillanuevaalicante/?pg=3) ['zyte-api']
    • latest_scrape

      2026-03-30 15:03:40 [scrapy.core.scraper] DEBUG: Scraped from <200 https://web.gw.coches.net/details/60727750>
    • latest_stat

      2026-03-30 15:03:37 [scrapy.extensions.logstats] INFO: Crawled 99 pages (at 12 pages/min), scraped 94 items (at 11 items/min)
    • Head

      2026-03-30 14:58:37 [scrapy.utils.log] INFO: Scrapy 2.11.2 started (bot: SourcingV2)
      2026-03-30 14:58:37 [scrapy.utils.log] INFO: Versions: lxml 5.2.2.0, libxml2 2.12.6, cssselect 1.2.0, parsel 1.9.1, w3lib 2.1.2, Twisted 24.3.0, Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0], pyOpenSSL 24.1.0 (OpenSSL 3.2.1 30 Jan 2024), cryptography 42.0.7, Platform Linux-5.15.0-1098-azure-x86_64-with-glibc2.36
      2026-03-30 14:58:37 [auto_selling_coches.net.api] INFO: Starting auto_selling_coches.net.api spider (API-based)
      2026-03-30 14:58:37 [auto_selling_coches.net.api] INFO: Extracted dealer slug: autosalonvillanuevaalicante
      2026-03-30 14:58:37 [auto_selling_coches.net.api] INFO: _job: 4a3f36662c1f11f18e133aacc95538c6, SCRAPY_JOB: None
      2026-03-30 14:58:37 [scrapy.addons] INFO: Enabled addons:
      []
      2026-03-30 14:58:37 [asyncio] DEBUG: Using selector: EpollSelector
      2026-03-30 14:58:37 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.asyncioreactor.AsyncioSelectorReactor
      2026-03-30 14:58:37 [scrapy.utils.log] DEBUG: Using asyncio event loop: asyncio.unix_events._UnixSelectorEventLoop
      2026-03-30 14:58:37 [scrapy.extensions.telnet] INFO: Telnet Password: 4d9b6cfeb54682b6
      2026-03-30 14:58:37 [scrapy.middleware] INFO: Enabled extensions:
      ['scrapy.extensions.corestats.CoreStats',
       'scrapy.extensions.telnet.TelnetConsole',
       'scrapy.extensions.memusage.MemoryUsage',
       'scrapy.extensions.feedexport.FeedExporter',
       'scrapy.extensions.logstats.LogStats',
       'scrapy.extensions.closespider.CloseSpider']
      2026-03-30 14:58:37 [scrapy.crawler] INFO: Overridden settings:
      {'BOT_NAME': 'SourcingV2',
       'CLOSESPIDER_TIMEOUT': 3600,
       'CONCURRENT_REQUESTS': 5,
       'DOWNLOAD_MAXSIZE': 52428800,
       'DOWNLOAD_WARNSIZE': 10485760,
       'FEED_EXPORT_ENCODING': 'utf-8',
       'LOG_FILE': '/var/log/scrapyd/logs/sourcing_v2/auto_selling_coches.net.api/4a3f36662c1f11f18e133aacc95538c6.log',
       'LOG_FORMATTER': 'crawlers.log_formatter.SourcingLogFormatter',
       'MEMUSAGE_LIMIT_MB': 2048,
       'MEMUSAGE_WARNING_MB': 1536,
       'NEWSPIDER_MODULE': 'spiders',
       'REQUEST_FINGERPRINTER_CLASS': 'scrapy_zyte_api.ScrapyZyteAPIRequestFingerprinter',
       'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7',
       'SPIDER_MODULES': ['spiders', 'auth_check'],
       'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor',
       'USER_AGENT': ''}
      2026-03-30 14:58:37 [scrapy_zyte_api.handler] INFO: Using a Zyte API key starting with '5857011'
      2026-03-30 14:58:37 [scrapy_zyte_api.handler] INFO: Using a Zyte API key starting with '5857011'
      2026-03-30 14:58:37 [scrapy.middleware] INFO: Enabled downloader middlewares:
      ['scrapy.downloadermiddlewares.offsite.OffsiteMiddleware',
       'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
       'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
       'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
       'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
       'scrapy_zyte_api.ScrapyZyteAPIDownloaderMiddleware',
       'scrapy.downloadermiddlewares.retry.RetryMiddleware',
       'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
       'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
       'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
       'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
       'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
       'scrapy.downloadermiddlewares.stats.DownloaderStats']
      2026-03-30 14:58:37 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Setting up IdGenerationMiddleware for auto_selling
      2026-03-30 14:58:37 [scrapy.middleware] INFO: Enabled spider middlewares:
      ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
       'scrapy_zyte_api.ScrapyZyteAPISpiderMiddleware',
       'scrapy.spidermiddlewares.referer.RefererMiddleware',
       'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
       'scrapy.spidermiddlewares.depth.DepthMiddleware',
       'crawlers.middlewares.photo_download_middleware.PhotoDownloadMiddleware',
       'crawlers.auto_selling.middlewares.id_gen_middleware.IdGenMiddleware']
      2026-03-30 14:58:37 [scrapy.middleware] INFO: Enabled item pipelines:
      ['crawlers.pipelines.post_to_azurebus_auto_selling.AzureBusPipelineAutoSelling',
       'crawlers.pipelines.save_to_cdlq_db_auto_selling.SaveToCdlqDbAutoSelling']
      2026-03-30 14:58:37 [scrapy.core.engine] INFO: Spider opened
      2026-03-30 14:58:37 [azure.identity._credentials.environment] INFO: Incomplete environment configuration for EnvironmentCredential. These variables are set: AZURE_TENANT_ID, AZURE_CLIENT_ID
      2026-03-30 14:58:37 [common.service_bus.service_bus_client_factory] INFO: ServiceBusClient initialized via DefaultAzureCredential for alx-production-bus.servicebus.windows.net
      2026-03-30 14:58:37 [crawlers.pipelines.base_azure_bus_pipeline] INFO: [AzureBusPipeline] Connected to topic auto.sourcing-v2
      2026-03-30 14:58:37 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
      2026-03-30 14:58:37 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6024
      2026-03-30 14:58:37 [scrapy.extensions.memusage] INFO: Peak memory usage is 142MiB
      2026-03-30 14:58:37 [scrapy-playwright] INFO: Starting download handler
      
      2026-03-30 14:58:37 [scrapy-playwright] INFO: Starting download handler
      2026-03-30 14:58:42 [auto_selling_coches.net.api] INFO: API spider started with URL: https://www.coches.net/concesionario/autosalonvillanuevaalicante, dealer_slug: autosalonvillanuevaalicante
      2026-03-30 14:58:42 [auto_selling_coches.net.api] INFO: Trying fast dealer API: https://web.gw.coches.net/dealers/autosalonvillanuevaalicante
      2026-03-30 14:58:42 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 1st time calling it.
      2026-03-30 14:58:43 [scrapy.core.engine] DEBUG: Crawled (400) <GET https://web.gw.coches.net/dealers/autosalonvillanuevaalicante> (referer: None) ['zyte-api']
      2026-03-30 14:58:43 [auto_selling_coches.net.api] INFO: Search API failed (Ignoring non-200 response), falling back to browserHtml
      2026-03-30 14:58:43 [auto_selling_coches.net.api] INFO: Using browserHtml fallback path (page 1)
      
      2026-03-30 14:58:43 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 1st time calling it.
      2026-03-30 14:59:12 [zyte_api._retry] DEBUG: Finished call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request' after 28.217(s), this was the 1st time calling it.
      2026-03-30 14:59:12 [zyte_api._retry] DEBUG: Retrying zyte_api._async.AsyncZyteAPI.get.<locals>.request in 5.068460378816454 seconds as it raised RequestError: RequestError: 520, message=, headers=<CIMultiDictProxy('Date': 'Mon, 30 Mar 2026 14:59:11 GMT', 'Content-Type': 'application/problem+json', 'Content-Length': '252', 'Connection': 'keep-alive', 'Retry-After': '60', 'Request-Id': '839d8c30550a3d643f3e4806dabf1719', 'Vary': 'Accept-Encoding', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload', 'Access-Control-Allow-Credentials': 'true', 'Access-Control-Expose-Headers': '*, X-Project-Key', 'Access-Control-Allow-Methods': 'GET, PUT, POST, DELETE, PATCH, OPTIONS', 'Access-Control-Allow-Headers': 'DNT, Keep-Alive, User-Agent, X-Requested-With, If-Modified-Since, Cache-Control, Content-Type, Range, Authorization, X-Project-Key', 'Access-Control-Max-Age': '1728000')>, body=b'{"type":"/download/website-ban","title":"Website Ban","status":520,"detail":"Zyte API could not get a ban-free response in a reasonable time. Please, check your URL query parameters. See https://docs.zyte.com/zyte-api/usage/errors.html#zapi-error-url"}', request_id=839d8c30550a3d643f3e4806dabf1719.
      
      2026-03-30 14:59:17 [zyte_api._retry] DEBUG: Starting call to 'zyte_api._async.AsyncZyteAPI.get.<locals>.request', this is the 2nd time calling it.
      2026-03-30 14:59:37 [scrapy.extensions.logstats] INFO: Crawled 1 pages (at 1 pages/min), scraped 0 items (at 0 items/min)
      
      2026-03-30 14:59:37 [scrapy.extensions.memusage] INFO: Peak memory usage is 151MiB
      2026-03-30 15:00:37 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.coches.net/concesionario/autosalonvillanuevaalicante?pg=1> (referer: None) ['zyte-api']
      2026-03-30 15:00:37 [auto_selling_coches.net.api] INFO: Total items from HTML header: 96
      2026-03-30 15:00:37 [auto_selling_coches.net.api] INFO: Extracting listings from HTML page 1
      2026-03-30 15:00:37 [auto_selling_coches.net.api] INFO: Found 30 listings in HTML (page 1)
      2026-03-30 15:00:37 [auto_selling_coches.net.api] INFO: Found listing with ID: 70282627
      2026-03-30 15:00:37 [azure.identity._credentials.environment] INFO: Incomplete environment configuration for EnvironmentCredential. These variables are set: AZURE_TENANT_ID, AZURE_CLIENT_ID
      2026-03-30 15:00:37 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): login.microsoftonline.com:443
      2026-03-30 15:00:37 [urllib3.connectionpool] DEBUG: https://login.microsoftonline.com:443 "POST /8ea908c1-4e85-4692-bc3f-3646b9b40891/oauth2/v2.0/token HTTP/1.1" 200 2113
      2026-03-30 15:00:37 [azure.identity._credentials.chained] INFO: DefaultAzureCredential acquired a token from WorkloadIdentityCredential
      2026-03-30 15:00:37 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): alxsourcingstorageprod.table.core.windows.net:443
      2026-03-30 15:00:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
      2026-03-30 15:00:37 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_coches.net.api%27%20and%20RowKey%20eq%20%2770282627%27 HTTP/1.1" 200 None
      2026-03-30 15:00:37 [auto_selling_coches.net.api] INFO: Scrape type for 70282627: 1
      2026-03-30 15:00:37 [auto_selling_coches.net.api] INFO: Scraping new listing 70282627
      2026-03-30 15:00:37 [scrapy.extensions.logstats] INFO: Crawled 2 pages (at 1 pages/min), scraped 0 items (at 0 items/min)
      2026-03-30 15:00:37 [scrapy.extensions.memusage] INFO: Peak memory usage is 165MiB
    • Tail

       'registration_date': '2022-01-01',
       'seats_number': 5,
       'seller_name': None,
       'title': 'BMW X3 M Competition',
       'transmission': 'automatic',
       'trim': 'M Competition',
       'vin': None,
       'year': 2022}
      2026-03-30 15:03:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
      2026-03-30 15:03:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_coches.net.api%27%20and%20RowKey%20eq%20%2762075223%27 HTTP/1.1" 200 None
      2026-03-30 15:03:38 [auto_selling_coches.net.api] INFO: Saving data for 62075223: {'created_time': 1774883018.939025, 'last_price_update_time': 1774883018.939035}
      2026-03-30 15:03:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
      2026-03-30 15:03:38 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_coches.net.api',RowKey='62075223') HTTP/1.1" 204 0
      2026-03-30 15:03:40 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://web.gw.coches.net/details/60727750> (referer: https://www.coches.net/concesionario/autosalonvillanuevaalicante/?pg=3) ['zyte-api']
      2026-03-30 15:03:40 [auto_selling_coches.net.api] INFO: Extracted item from API: 60727750 - MASERATI Grecale GT L4 MHEV AWD
      2026-03-30 15:03:40 [crawlers.auto_selling.middlewares.id_gen_middleware] INFO: Generated ID for item: 62155ad5-4172-59a1-a916-f263da0aac94 with identifier: 60727750
      2026-03-30 15:03:40 [crawlers.middlewares.photo_download_middleware] INFO: Downloading 1 photos for item 62155ad5-4172-59a1-a916-f263da0aac94
      2026-03-30 15:03:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
      2026-03-30 15:03:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /PhotoDownloadCache()?$filter=PartitionKey%20eq%20%27auto_selling_coches.net.api%27%20and%20RowKey%20eq%20%27https%3A%252F%252Fa.ccdn.es%252Fcnet%252Fvehicles%252F17953708%252Fbca77072-cb01-444c-ba79-7eab21a78843.jpg%27 HTTP/1.1" 200 None
      2026-03-30 15:03:40 [crawlers.middlewares.photo_download_middleware] INFO: Image https://a.ccdn.es/cnet/vehicles/17953708/bca77072-cb01-444c-ba79-7eab21a78843.jpg already downloaded
      2026-03-30 15:03:40 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: Azure Bus Pipeline processing item: 62155ad5-4172-59a1-a916-f263da0aac94 with identifier 60727750
      2026-03-30 15:03:40 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
      2026-03-30 15:03:40 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
      2026-03-30 15:03:40 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process item CloudEvent for item 62155ad5-4172-59a1-a916-f263da0aac94
      2026-03-30 15:03:40 [scrapy.core.scraper] DEBUG: Scraped from <200 https://web.gw.coches.net/details/60727750>
      {'auction_title': '',
       'body_type': 'SUV',
       'c_o_2_emission_value': None,
       'category': 'Coches',
       'color': 'Negro',
       'currency': 'EUR',
       'description': 'MASERATI GRECALE GT 2.0 300CV\n'
                      '\n'
                      'Cambio automático con levas en el volante\n'
                      'Llantas Forja Etere de 20”\n'
                      'Faros principales adaptativos de matriz LED con luces diurnas '
                      'LED integradas y luces de curvas\n'
                      'y luces traseras LED\n'
                      'Asientos de cuero eléctricos , calefactables, con ajuste '
                      'lumbar\n'
                      'Asientos ajustables eléctricamente en 14 posiciones con \n'
                      'Acristalamiento de privacidad en la parte trasera\n'
                      'Espejos exteriores con atenuación automática\n'
                      'Volante de cuero de 3 radios\n'
                      'Cámara trasera\n'
                      'Pantalla táctil de 12,3 pulgadas con sistema de navegación y '
                      'pantalla de confort integrada de 8,8 pulgadas\n'
                      'Sistema de sonido premium Sonus Faber\n'
                      'Portón trasero eléctrico\n'
                      'Sistema de alarma de seguridad\n'
                      'Radio digital\n'
                      'Asistente de velocidad inteligente\n'
                      'Reconocimiento de señales de tráfico\n'
                      'Asistente de conducción activo\n'
                      'Aviso activo de cambio de carril con asistente de punto '
                      'ciego\n'
                      'Detección de fatiga\n'
                      'Base de carga por inducción RFX\n'
                      'Chasis Skyhook con suspensión adaptativa\n'
                      'Pedales deportivos 4CS fabricados en acero inoxidable.\n'
                      'Pinzas de freno de aluminio pintadas de rojo\n'
                      'y más extras...\n'
                      '\n'
                      '\n'
                      '-1 año de garantía y transferencia de nombre incluida en el '
                      'precio.\n'
                      '-Consulte nuestras opciones de financiación si lo desea (el '
                      'precio no está sujeto a financiar con nosotros '
                      'directamente).\n'
                      '-Servicio de entrega a domicilio en cualquier punto de la '
                      'Península y Baleares, precio a consultar.\n'
                      '-Solicite prueba de vehículo a domicilio.\n'
                      '-Todos nuestros vehículos disponen de libro de mantenimiento '
                      'y certificación de kilometraje.\n'
                      '-Atendemos con cita previa, contacte con nosotros para '
                      'solicitar una prueba del vehículo.\n'
                      '-Este anuncio no es vinculante, puede estar sujeto a errores, '
                      'se muestra a titulo informativo y no contractual.\n'
                      '-19 años de experiencia nos avalan en el sector del '
                      'automóvil.\n'
                      '-We speak English',
       'doors_number': 5,
       'emission_standard': None,
       'engine': '2.0L',
       'engine_horse_power': 300,
       'expiration_date': None,
       'fuel_type': 'Gasolina',
       'id': '62155ad5-4172-59a1-a916-f263da0aac94',
       'is_damaged': False,
       'is_operable': True,
       'license_plate': None,
       'link_to_web_offer': 'https://www.coches.net/maserati-grecale-gt-l4-mhev-300cv-awd-5p-gasolina-2022-en-alicante-60727750-covo.aspx',
       'listing': 'auction',
       'location': 'Alicante',
       'make': 'MASERATI',
       'model': 'Grecale',
       'odometer': 19000,
       'odometer_reading_unit': 'km',
       'origin_country_code': 'ES',
       'original_photo_count': 1,
       'photo_urls': ['https://a.ccdn.es/cnet/vehicles/17953708/bca77072-cb01-444c-ba79-7eab21a78843.jpg'],
       'photos': [{'BlobStorageUrl': 'https://alxproduction.blob.core.windows.net/top-deal-listing/listing_62155ad5-4172-59a1-a916-f263da0aac94/62155ad5-4172-59a1-a916-f263da0aac94_image_0.jpg',
                   'Extension': 'jpg',
                   'FileSizeInBytes': 230029,
                   'Height': 600,
                   'OriginUrl': 'https://a.ccdn.es/cnet/vehicles/17953708/bca77072-cb01-444c-ba79-7eab21a78843.jpg',
                   'Width': 800}],
       'price': 68500,
       'price_includes_vat': True,
       'registration_date': '2022-01-01',
       'seats_number': 5,
       'seller_name': None,
       'title': 'MASERATI Grecale GT L4 MHEV AWD',
       'transmission': 'automatic',
       'trim': 'GT L4 MHEV 300CV AWD',
       'vin': None,
       'year': 2022}
      2026-03-30 15:03:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
      2026-03-30 15:03:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_coches.net.api%27%20and%20RowKey%20eq%20%2760727750%27 HTTP/1.1" 200 None
      2026-03-30 15:03:40 [auto_selling_coches.net.api] INFO: Saving data for 60727750: {'created_time': 1774883020.544431, 'last_price_update_time': 1774883020.544442}
      2026-03-30 15:03:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
      2026-03-30 15:03:40 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "PATCH /ScrapedListings(PartitionKey='auto_selling_coches.net.api',RowKey='60727750') HTTP/1.1" 204 0
      2026-03-30 15:03:40 [scrapy.core.engine] INFO: Closing spider (finished)
      2026-03-30 15:03:40 [crawlers.pipelines.save_to_cdlq_db_auto_selling] DEBUG: [SaveToCdlqDbAutoSelling] failed to close cursor: 'SaveToCdlqDbAutoSelling' object has no attribute 'cursor'
      2026-03-30 15:03:40 [crawlers.pipelines.save_to_cdlq_db_auto_selling] DEBUG: [SaveToCdlqDbAutoSelling] failed to close connection: 'SaveToCdlqDbAutoSelling' object has no attribute 'conn'
      2026-03-30 15:03:40 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
      2026-03-30 15:03:40 [azure.servicebus._pyamqp.cbs] DEBUG: CBS status check: state == <CbsAuthState.OK: 0>, expired == False, refresh required == False
      2026-03-30 15:03:40 [crawlers.pipelines.post_to_azurebus_auto_selling] DEBUG: [AzureBusPipeline] sent process scrape complete CloudEvent for item 4a3f36662c1f11f18e133aacc95538c6
      2026-03-30 15:03:40 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
      2026-03-30 15:03:40 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
      2026-03-30 15:03:40 [azure.servicebus._pyamqp.management_link] INFO: Management link receiver state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
      2026-03-30 15:03:40 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
      2026-03-30 15:03:40 [azure.servicebus._pyamqp.management_link] INFO: Management link sender state changed: <LinkState.ATTACHED: 3> -> <LinkState.DETACH_SENT: 4>
      2026-03-30 15:03:40 [azure.servicebus._pyamqp.session] INFO: Session state changed: <SessionState.MAPPED: 3> -> <SessionState.END_SENT: 4>
      2026-03-30 15:03:40 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.OPENED: 9> -> <ConnectionState.CLOSE_SENT: 11>
      2026-03-30 15:03:40 [azure.servicebus._pyamqp._connection] INFO: Connection state changed: <ConnectionState.CLOSE_SENT: 11> -> <ConnectionState.END: 13>
      2026-03-30 15:03:40 [azure.servicebus._pyamqp.session] INFO: Session state changed: <SessionState.END_SENT: 4> -> <SessionState.DISCARDING: 6>
      2026-03-30 15:03:40 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
      2026-03-30 15:03:40 [azure.servicebus._pyamqp.management_link] INFO: Management link sender state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
      2026-03-30 15:03:40 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
      2026-03-30 15:03:40 [azure.servicebus._pyamqp.management_link] INFO: Management link receiver state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
      2026-03-30 15:03:40 [azure.servicebus._pyamqp.link] INFO: Link state changed: <LinkState.DETACH_SENT: 4> -> <LinkState.DETACHED: 0>
      2026-03-30 15:03:41 [crawlers.pipelines.post_to_azurebus_auto_selling] INFO: [AzureBusPipeline] connection closed
      2026-03-30 15:03:41 [auto_selling_coches.net.api] INFO: auto_selling_coches.net.api Crawl ended with reason finished, scrape types: {<ScrapeType.NEW: 1>: 6, <ScrapeType.NEW_DUPLICATE_ID: 4>: 0, <ScrapeType.PRICE_UPDATE: 2>: 0, <ScrapeType.AUCTION_UPDATE: 3>: 0, <ScrapeType.SKIPPED: 0>: 0}
      2026-03-30 15:03:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "POST /Tables HTTP/1.1" 409 None
      2026-03-30 15:03:41 [urllib3.connectionpool] DEBUG: https://alxsourcingstorageprod.table.core.windows.net:443 "GET /ScrapedListings()?$filter=PartitionKey%20eq%20%27auto_selling_coches.net.api%27%20and%20last_price_update_time%20lt%201773587021 HTTP/1.1" 200 None
      2026-03-30 15:03:41 [scrapy.extensions.feedexport] INFO: Stored jsonlines feed (96 items) in: file:///var/lib/scrapyd/items/sourcing_v2/auto_selling_coches.net.api/4a3f36662c1f11f18e133aacc95538c6.jl
      2026-03-30 15:03:41 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
      {'downloader/request_bytes': 199044,
       'downloader/request_count': 101,
       'downloader/request_method_count/GET': 101,
       'downloader/response_bytes': 9021218,
       'downloader/response_count': 101,
       'downloader/response_status_count/200': 100,
       'downloader/response_status_count/400': 1,
       'elapsed_time_seconds': 304.103178,
       'feedexport/success_count/FileFeedStorage': 1,
       'finish_reason': 'finished',
       'finish_time': datetime.datetime(2026, 3, 30, 15, 3, 41, 787922, tzinfo=datetime.timezone.utc),
       'item_scraped_count': 96,
       'log_count/DEBUG': 1382,
       'log_count/INFO': 951,
       'memusage/max': 194498560,
       'memusage/startup': 149626880,
       'request_depth_max': 5,
       'response_received_count': 101,
       'scheduler/dequeued': 101,
       'scheduler/dequeued/memory': 101,
       'scheduler/enqueued': 101,
       'scheduler/enqueued/memory': 101,
       'scrape_type/new': 96,
       'scrapy-zyte-api/429': 0,
       'scrapy-zyte-api/attempts': 103,
       'scrapy-zyte-api/error_ratio': 0.019417475728155338,
       'scrapy-zyte-api/error_types/download/website-ban': 2,
       'scrapy-zyte-api/errors': 2,
       'scrapy-zyte-api/fatal_errors': 0,
       'scrapy-zyte-api/mean_connection_seconds': 7.819243302677296,
       'scrapy-zyte-api/mean_response_seconds': 7.470479665748259,
       'scrapy-zyte-api/processed': 101,
       'scrapy-zyte-api/request_args/actions': 4,
       'scrapy-zyte-api/request_args/browserHtml': 4,
       'scrapy-zyte-api/request_args/customHttpRequestHeaders': 97,
       'scrapy-zyte-api/request_args/experimental.requestCookies': 100,
       'scrapy-zyte-api/request_args/experimental.responseCookies': 101,
       'scrapy-zyte-api/request_args/httpResponseBody': 97,
       'scrapy-zyte-api/request_args/httpResponseHeaders': 97,
       'scrapy-zyte-api/request_args/requestHeaders': 3,
       'scrapy-zyte-api/request_args/url': 101,
       'scrapy-zyte-api/status_codes/200': 101,
       'scrapy-zyte-api/status_codes/520': 2,
       'scrapy-zyte-api/success': 101,
       'scrapy-zyte-api/success_ratio': 1.0,
       'scrapy-zyte-api/throttle_ratio': 0.0,
       'start_time': datetime.datetime(2026, 3, 30, 14, 58, 37, 684744, tzinfo=datetime.timezone.utc)}
      2026-03-30 15:03:41 [scrapy.core.engine] INFO: Spider closed (finished)
      2026-03-30 15:03:41 [asyncio] ERROR: Unclosed client session
      client_session: <aiohttp.client.ClientSession object at 0x7f1054122f50>
      2026-03-30 15:03:41 [asyncio] ERROR: Unclosed client session
      client_session: <aiohttp.client.ClientSession object at 0x7f1054123790>
      
    • Log

      /2/log/utf8/sourcing_v2/auto_selling_coches.net.api/4a3f36662c1f11f18e133aacc95538c6/?job_finished=True&ui=mobile

    • Source

      http://scrapyd-1:6800/logs/sourcing_v2/auto_selling_coches.net.api/4a3f36662c1f11f18e133aacc95538c6.log

  • sourcelog
    last update time2026-03-30 15:03:41
    last update timestamp1774883021
    downloader / request bytes199044
    downloader / request count101
    downloader / request method count / GET101
    downloader / response bytes9021218
    downloader / response count101
    downloader / response status count / 200100
    downloader / response status count / 4001
    elapsed time seconds304.103178
    feedexport / success count / FileFeedStorage1
    finish reasonfinished
    finish timedatetime.datetime(2026, 3, 30, 15, 3, 41, 787922, tzinfo=datetime.timezone.utc)
    item scraped count96
    log count / DEBUG1382
    log count / INFO951
    memusage / max194498560
    memusage / startup149626880
    request depth max5
    response received count101
    scheduler / dequeued101
    scheduler / dequeued / memory101
    scheduler / enqueued101
    scheduler / enqueued / memory101
    scrape type / new96
    scrapy-zyte-api / 4290
    scrapy-zyte-api / attempts103
    scrapy-zyte-api / error ratio0.019417475728155338
    scrapy-zyte-api / error types / download / website-ban2
    scrapy-zyte-api / errors2
    scrapy-zyte-api / fatal errors0
    scrapy-zyte-api / mean connection seconds7.819243302677296
    scrapy-zyte-api / mean response seconds7.470479665748259
    scrapy-zyte-api / processed101
    scrapy-zyte-api / request args / actions4
    scrapy-zyte-api / request args / browserHtml4
    scrapy-zyte-api / request args / customHttpRequestHeaders97
    scrapy-zyte-api / request args / experimental.requestCookies100
    scrapy-zyte-api / request args / experimental.responseCookies101
    scrapy-zyte-api / request args / httpResponseBody97
    scrapy-zyte-api / request args / httpResponseHeaders97
    scrapy-zyte-api / request args / requestHeaders3
    scrapy-zyte-api / request args / url101
    scrapy-zyte-api / status codes / 200101
    scrapy-zyte-api / status codes / 5202
    scrapy-zyte-api / success101
    scrapy-zyte-api / success ratio1.0
    scrapy-zyte-api / throttle ratio0.0
    start timedatetime.datetime(2026, 3, 30, 14, 58, 37, 684744, tzinfo=datetime.timezone.utc)