[Bug]: The class attribute extracted by crawl4ai is wrong

[mobsceneZ]

crawl4ai version

0.8.0

Expected Behavior

We should be able to select the corresponding element using the css selector "a span.fn".

Current Behavior

The following minimal poc is used to describe the problem:

import asyncio
from crawl4ai import AsyncWebCrawler, JsonCssExtractionStrategy
from crawl4ai.async_configs import BrowserConfig, CrawlerRunConfig

schema = {
    "name": "minimal reproducer",
    "baseSelector": "td.change-author",
    "type": "nested_list",
    "fields": [
        {"name": "field1", "selector": "a span", "type": "text"},
        {"name": "field2", "selector": "a span", "type": "attribute", "attribute": "class"},
        {"name": "field3", "selector": "a span.fn", "type": "text"},
    ]
}

async def main():
    browser_config = BrowserConfig()
    crawler_config = CrawlerRunConfig(extraction_strategy=JsonCssExtractionStrategy(schema))

    async with AsyncWebCrawler(config=browser_config) as web_crawler:
        result = await web_crawler.arun(
            url="https://bugzilla.mozilla.org/show_bug.cgi?id=1770266",
            config=crawler_config
        )
        
        if result.success:
            print(result.extracted_content)

if __name__ == "__main__":
    asyncio.run(main())

Running the above poc gives the result:

$ python3 minimal_poc.py
[INIT].... → Crawl4AI 0.8.0 
[FETCH]... ↓ https://bugzilla.mozilla.org/show_bug.cgi?id=1770266                                                 | ✓ | ⏱: 2.76s 
[SCRAPE].. ◆ https://bugzilla.mozilla.org/show_bug.cgi?id=1770266                                                 | ✓ | ⏱: 0.03s 
[EXTRACT]. ■ https://bugzilla.mozilla.org/show_bug.cgi?id=1770266                                                 | ✓ | ⏱: 0.03s 
[COMPLETE] ● https://bugzilla.mozilla.org/show_bug.cgi?id=1770266                                                 | ✓ | ⏱: 2.83s 
[
    {
        "field1": "Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)",
        "field2": [
            "fna"
        ]
    },
    {
        "field1": "Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)",
        "field2": [
            "fna"
        ]
    },
...

We cannot extract the corresponding element using css selector "a span.fn". Instead, by inspecting the class attribute of "a span", we find that it contains a weird value "fna" instead of "fn".

Is this reproducible?

Yes

OS

macOS

Python version

3.14.3

Browser

Chrome

Browser version

No response

Error logs & Screenshots (if applicable)

No response

[hafezparast]

Hi @mobsceneZ — thanks for the detailed report and reproduction case!

I investigated this and it turns out the page's HTML actually uses class="fna", not class="fn". This is the case at every level:

1. Raw server HTML — requests.get() returns <span class="fna">Gary Kwong...</span>
2. Playwright rendered DOM — page.query_selector_all('a span.fna') returns 17 results, 'a span.fn' returns 0
3. All span classes on the page — there is no instance of class="fn" anywhere in the HTML

You can verify this yourself by viewing the page source (Ctrl+U) and searching for class="fn" vs class="fna".

It looks like Bugzilla uses fna instead of the standard hCard fn (formatted name) class. So crawl4ai is correctly extracting the attribute as it exists in the HTML.

To fix your schema, change:

{"name": "field3", "selector": "a span.fna", "type": "text"},

Hope that helps! Closing this as not a crawl4ai bug.

@unclecode @ntohidi this can be closed.

[ntohidi]

I will close the issue, but feel free to continue the conversation :)

[mobsceneZ]

Hi @mobsceneZ — thanks for the detailed report and reproduction case!

I investigated this and it turns out the page's HTML actually uses class="fna", not class="fn". This is the case at every level:

1. Raw server HTML — requests.get() returns <span class="fna">Gary Kwong...</span>
2. Playwright rendered DOM — page.query_selector_all('a span.fna') returns 17 results, 'a span.fn' returns 0
3. All span classes on the page — there is no instance of class="fn" anywhere in the HTML

You can verify this yourself by viewing the page source (Ctrl+U) and searching for class="fn" vs class="fna".

It looks like Bugzilla uses fna instead of the standard hCard fn (formatted name) class. So crawl4ai is correctly extracting the attribute as it exists in the HTML.

To fix your schema, change:

{"name": "field3", "selector": "a span.fna", "type": "text"},
Hope that helps! Closing this as not a crawl4ai bug.

@unclecode @ntohidi this can be closed.

Thanks for your response @hafezparast ! After reading through your comment, I have tried through different browsers and platforms and finally found out that the problem is with whether you're logged in. After I log out, this element becomes <span class="fna">, which is what we would expect. Sorry for making a silly mistake and trully appreciate your kind response!

[hafezparast]

No worries at all @mobsceneZ! Login-dependent class names are a common gotcha when scraping. Thanks for digging into it and sharing — this could help others hitting similar issues with authenticated vs anonymous page differences. Happy crawling! 🔥