How to create a Filing object from a local SGML file

[dsdole]

Hi,
First off I'd like to say this library is really impressive. I've been wanting to try and learn/implement much of this functionality for years but never got around to it so thanks so much!

Anyway, I am trying to build some data pipelines for EDGAR and in order to avoid running into rate-limits during queries I'm trying to build a local mirror of the SGML submission files for the companies and forms I want.

I have a full local index of every filing in EDGAR, and a local mirror containing every SGML submission I want. Now I'm working on parsing the local SGML files. I'd like to create Filing objects from the SGML files but I can't figure out how to convert the FilingSGML' objects to a Filing`.

I'm sure the functionality exists here but maybe it's not as simple as I'm thinking. Any ideas?

[dsdole]

Ok, I'd like to refine my question:

Turns out the Filing.from_sgml() method works to create a Filing from the sgml text file, but even then it requires api calls for tickers.txt and when i try to call Filing.xbrl() it still requires the user agent be set. Is it possible to use the Filing object without making external requests at all? My end goal is to extract what ever information I can get for hundreds of companies and load it into a database for frequent, interactive analysis.

[dgunning]

You want to turn Local Storage ON and download data

use_local_storage() # OR os.environ['EDGAR_USE_LOCAL_DATA'] = 1
download_edgar_data()

[dsdole]

So is there no way to have the library use files which have already been downloaded by an external system? I am currently using some internal functions to build a full index and download sgmls in dagster pipelines.

# I want to do something like this:
from edgar.sgml import FilingSGML
from edgar import Filing
from pathlib import Path

file = './0000320193-25-000057.txt'

sgml = FilingSGML.from_source(Path(file))

tenq = Filing.from_sgml('0000320193-25-000057.txt')

Is the better(or only) way to avoid any external requests to instead set the $EDGAR_LOCAL_DATA_DIR variable to the location I want and then use the existing local storage functionality like that? I suppose this may be a bit of an X-Y problem: My end goal is this: once all necessary data is downloaded, I want to execute many concurrent pipeline tasks to quickly process the data into tabular formats(where possible). If those concurrent tasks each need to make a few calls to the API, then I'll hit the rate limit pretty quickly.

I'm now thinking that I can concurrently feed the specific accession numbers to the library and then build my tables:

Filing(
    cik: int,
    company: str, 
    form: str,
    filing_date: str,
    accession_no: str)

I really appreciate your help!

[dgunning]

I just released a fix in Release 4.4.3 that eliminated 2 network calls that were still being issued when local storage was turned on. I just tested with my wifi off that I can create a filing from an SGML file

use_local_storage()

filing = Filing.from_sgml("0001104659-25-002604.txt")

So for your purposes you need to call use_local_storage and the basic functionality should work. You can download_edgar_data() to get more data locally (like tickers, submissions etc)

Let me know how I can help - I'm trying to get it working seamlessly

Local Storage documentation

[dsdole]

Wow, I updated and turned disconnected from the network, and it seems to work perfectly. This is really great!

filing = Filing.from_sgml(Path('./0000320193-25-000057.txt'))
filing.xbrl().statements.balance_sheet()

�[3m                    Consolidated Balance Sheets (Standardized)                     �[0m
�[3m       �[0m�[1;3mSecond Quarter Ended�[0m�[3m �[0m�[3m(In millions, except shares and per share data)�[0m�[3m        �[0m
                                                                                   
 �[1m �[0m�[1m                                                 �[0m�[1m �[0m �[1m �[0m�[1mMar 29, 2025�[0m�[1m �[0m �[1m �[0m�[1mSep 28, 2024�[0m�[1m �[0m 
 ───────────────────────────────────────────────────────────────────────────────── 
    ASSETS:                                                                        
      Current assets:                                                              
        Cash and Cash Equivalents                          $28,162        $29,943  
        Current Marketable Securities                      $20,336        $35,228  
        Accounts Receivable                                $26,136        $33,410  
        Vendor non-trade receivables                       $23,662        $32,833  
        Inventory                                           $6,269         $7,286  
        Other Current Assets                               $14,109        $14,287  
        Total Current Assets                              $118,674       $152,987  
      Non-current assets:                                                          
        Non Current Marketable Securities                  $84,424        $91,479  
        Property, Plant and Equipment                      $46,876        $45,680  
        Other Non Current Assets                           $81,259        $74,834  
        Total Non Current Assets                          $212,559       $211,993  
      Total Assets                                        $331,233       $364,980  
    LIABILITIES AND SHAREHOLDERS’ EQUITY:                                          
      Current liabilities:                                                         
        Accounts Payable                                   $54,126        $68,960  
        Other Current Liabilities                          $61,849        $78,304  
        Deferred revenue                                    $8,976         $8,249  
        Commercial paper                                    $5,982         $9,967  
        Short Term Debt                                    $13,638        $10,912  
        Total Current Liabilities                         $144,571       $176,392  
      Non-current liabilities:                                                     
        Long Term Debt                                     $78,566        $85,750  
        Other Non Current Liabilities                      $41,300        $45,888  
        Total Non Current Liabilities                     $119,866       $131,638  
      Total Liabilities                                   $264,437       $308,030  
      Commitments and contingencies                                                
      Shareholders’ equity:                                                        
        Common Stock                                       $88,711        $83,276  
        Retained Earnings                                $(15,552)      $(19,154)  
        Accumulated Other Comprehensive Income/Loss       $(6,363)       $(7,172)  
        Total Stockholders' Equity                         $66,796        $56,950  
      Total Liabilities and Stockholders' Equity          $331,233       $364,980  
                                                                                   

Thanks again!