Unable to retrieve data for some states using arc_read(): 'subscript out of bounds' error

[soheepark7]

Hi, first of all thank you for creating this r package. I am sharing issues below.

*Describe the bug
When trying to read data for certain states (like Alabama, Alaska, and others) from the ArcGIS service using arc_read(), I encounter the error Error in x[[1]] : subscript out of bounds. However, data for Michigan (state code 26) and Minnesota (state code 27) is retrieved successfully without errors.

*To Reproduce

michigan_26_geo <- arc_read(
"https://tigerweb.geo.census.gov/arcgis/rest/services/Generalized_ACS2023/Tracts_Blocks/MapServer/4",
where = "STATE = '26'" # Michigan (works fine)
)

alabama_1_geo <- arc_read(
"https://tigerweb.geo.census.gov/arcgis/rest/services/Generalized_ACS2023/Tracts_Blocks/MapServer/4",
where = "STATE = '1'" # Alabama (gives error)
)

alaska_2_geo <- arc_read(
"https://tigerweb.geo.census.gov/arcgis/rest/services/Generalized_ACS2023/Tracts_Blocks/MapServer/4",
where = "STATE = '2'" # Alaska (gives error)
)

minnesota_27_geo <- arc_read(
"https://tigerweb.geo.census.gov/arcgis/rest/services/Generalized_ACS2023/Tracts_Blocks/MapServer/4",
where = "STATE = '27'" # Minnesota (works fine)
)

*Expected behavior
I expected the function to retrieve data for all states without errors.

Additional context

The error Error in x[[1]] : subscript out of bounds occurs for certain states (Alabama, Alaska, and others), while data retrieval works correctly for Michigan and Minnesota.
The issue might be related to missing or incomplete data for certain states in the ArcGIS service.
I’m using the arcgislayers package in R for this operation.
Final Command
I ultimately want to fetch data for all states at once using the following command:

geo_all_acs2023 <- arc_read(
"https://tigerweb.geo.census.gov/arcgis/rest/services/Generalized_ACS2023/Tracts_Blocks/MapServer/4",
where = "STATE IN ('1', '2', '4', '5', '6', '8', '9', '10', '12', '13',
'15', '16', '17', '18', '19', '20', '21', '22', '23',
'24', '25', '26', '27', '28', '29', '30', '31', '32',
'33', '34', '35', '36', '37', '38', '39', '40', '41',
'42', '44', '45', '46', '47', '48', '49', '50', '51',
'53', '54', '55', '56')"
)

Thank you!

[JosiahParry]

Hi @soheepark7, thanks for the report.

This looks like to be a little bit of user error mixed with bad error messages on our part. Your state fips codes need to be two digits instead of one. See the below:

arc_open( "https://tigerweb.geo.census.gov/arcgis/rest/services/Generalized_ACS2023/Tracts_Blocks/MapServer/4") |> 
  arc_select(where = "STATE = '01'")

[JosiahParry]

The second issue is that the layer has a maximum page size of 10000 which is way too big—we should maybe intelligently set a different bound. So this ends up trying to get every single polygon in a single query which results in a time out.

This will accomplish what you want:

flayer <- arc_open("https://tigerweb.geo.census.gov/arcgis/rest/services/Generalized_ACS2023/Tracts_Blocks/MapServer/4")
  
geo_all_acs2023 <- arc_select(
    flayer,
    where = "STATE IN ('01', '02', '04', '05', '06', '08', '09', '10', '12', '13', '15', '16', '17', '18', '19', '20', '21', '22', '23',
    '24', '25', '26', '27', '28', '29', '30', '31', '32',
    '33', '34', '35', '36', '37', '38', '39', '40', '41',
    '42', '44', '45', '46', '47', '48', '49', '50', '51',
    '53', '54', '55', '56')",
    page_size = 1000
)

[soheepark7]

Hi @JosiahParry

Thank you so much for your time and helpful suggestions. I will try your advice and test the query again with the adjustments. Thanks again for your support!

[soheepark7]

Hello @JosiahParry
I tried again with the code you provided, but I’m getting the same error message - Error in x[[i]] : subscript out of bounds . Could you kindly look into this issue again?

I am sharing the code I entered and the error message as follows:

flayer <- arc_open("https://tigerweb.geo.census.gov/arcgis/rest/services/Generalized_ACS2023/Tracts_Blocks/MapServer/4")

View(flayer)
geo_all_acs2023 <- arc_select(

• flayer,
• where = "STATE IN ('01', '02', '04', '05', '06', '08', '09', '10', '12', '13', '15', '16', '17', '18', '19', '20', '21', '22', '23',

• '24', '25', '26', '27', '28', '29', '30', '31', '32',
  

• '33', '34', '35', '36', '37', '38', '39', '40', '41',
  

• '42', '44', '45', '46', '47', '48', '49', '50', '51',
  

• '53', '54', '55', '56')",
  

• page_size = 1000
• )
  Error in x[[i]] : subscript out of bounds

[soheepark7]

Since you suggested using two digits, I tried running only the '01' state separately as shown below, and it seems like the code is working fine! However, I’m still getting an error when trying to fetch multiple states.

*R code I used

al_01_geo <- arc_read(
"https://tigerweb.geo.census.gov/arcgis/rest/services/Generalized_ACS2023/Tracts_Blocks/MapServer/4",
where = "STATE = '01'"
)