Skip to content

Vector Data Visualization in R#494

Closed
HarshiniGirish wants to merge 22 commits intoMAAP-Project:developfrom
HarshiniGirish:Vector-data
Closed

Vector Data Visualization in R#494
HarshiniGirish wants to merge 22 commits intoMAAP-Project:developfrom
HarshiniGirish:Vector-data

Conversation

@HarshiniGirish
Copy link
Collaborator

@HarshiniGirish HarshiniGirish commented Apr 24, 2025

This notebook I referenced.
A lot of steps here is data extraction and data analysis. Do I keep everything or just reference the other notebook.

I don't know what else can be done in terms of visualization.

@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@wildintellect
Copy link
Collaborator

Overall I think the use of GEDI here makes the tutorial more complicated that it needs to be. I think we should try to use some data out of MAAP STAC and focus on the different visualization options since using Jupyter with R is a little unusual.

  • plot (for printing, include saving a plot to png)
  • leaflet
  • tmap (optional but likely a good choice)
  • ggplot (optional)

Source formats, we should probably include:

  • geojson
  • geopackage
  • geoparquet

https://docs.maap-project.org/en/develop/technical_tutorials/visualization/visualize_lonboard.html is the closest existing example. <- something is wrong with this notebook the plots are missing.

@hrodmn @chuckwondo can you think of good vector data to visualize? https://github.com/MAAP-Project/maap-documentation-examples/blob/main/gedi-subset/notebooks/lonboard_exploration.ipynb I think is too much for R

@hrodmn
Copy link
Contributor

hrodmn commented Apr 28, 2025

@hrodmn @chuckwondo can you think of good vector data to visualize?

I'm trying to think of data that would be useful to add to a MAAP-centric graphic.

Maybe an ecoregion type of dataset - this source has a zipped shapefile: https://www.worldwildlife.org/publications/terrestrial-ecoregions-of-the-world

Or maybe National Forest boundaries from USFS: https://data.fs.usda.gov/geodata/edw/datasets.php?dsetCategory=boundaries

In my forest science days I was always making maps of field plot locations which usually had a point at the center and a radius drawn around them.

There are not very many examples of Geoparquet out in the wild - maybe something like this? https://planetarycomputer.microsoft.com/dataset/noaa-climate-normals-tabular

@HarshiniGirish
Copy link
Collaborator Author

Up for review! @wildintellect @smk0033

@review-notebook-app
Copy link

review-notebook-app bot commented Aug 1, 2025

View / edit / reply to this conversation on ReviewNB

wildintellect commented on 2025-08-01T18:18:56Z
----------------------------------------------------------------

I think we also want a static plot example using plot, and tmap


@review-notebook-app
Copy link

review-notebook-app bot commented Aug 1, 2025

View / edit / reply to this conversation on ReviewNB

wildintellect commented on 2025-08-01T18:18:57Z
----------------------------------------------------------------

Need to put the sample data into a Shared bucket so the path works for everyone.


@review-notebook-app
Copy link

review-notebook-app bot commented Aug 1, 2025

View / edit / reply to this conversation on ReviewNB

wildintellect commented on 2025-08-01T18:18:57Z
----------------------------------------------------------------

I think we need to find a simpler way to read the data into sf.

the geoarrow package has a read_geoparquet_sf() function but I'm not sure if that's still the best practice

We should check if adding libgdal-arrow-parquet lets sf just work via the ogr driver too.


@HarshiniGirish
Copy link
Collaborator Author

thank you @wildintellect. Its up for another review!

@review-notebook-app
Copy link

review-notebook-app bot commented Aug 4, 2025

View / edit / reply to this conversation on ReviewNB

wildintellect commented on 2025-08-04T23:40:00Z
----------------------------------------------------------------

Circle Radius is way too big, try 50 or 100,

might be good to also include agbd and agbd_se in the popup


@review-notebook-app
Copy link

review-notebook-app bot commented Aug 4, 2025

View / edit / reply to this conversation on ReviewNB

wildintellect commented on 2025-08-04T23:40:00Z
----------------------------------------------------------------

Line #4.    df_sf <- st_as_sf(df, crs = 4326)

Don't repeat the conversion, use the same from above, conversion to SF should happen in the Load section. Load and convert to SF


@review-notebook-app
Copy link

review-notebook-app bot commented Aug 4, 2025

View / edit / reply to this conversation on ReviewNB

wildintellect commented on 2025-08-04T23:40:02Z
----------------------------------------------------------------

Remember AGBD is the single most important value in the data (would be nice to apply color ramp to it even)


@review-notebook-app
Copy link

review-notebook-app bot commented Aug 4, 2025

View / edit / reply to this conversation on ReviewNB

wildintellect commented on 2025-08-04T23:40:03Z
----------------------------------------------------------------

World isn't loaded/imported before this. Need to add data(World) just after the library imports in the top cell.

Also remember to move all Library calls to the top.


@HarshiniGirish
Copy link
Collaborator Author

Made the changes. Up for review @wildintellect

@review-notebook-app
Copy link

review-notebook-app bot commented Aug 5, 2025

View / edit / reply to this conversation on ReviewNB

wildintellect commented on 2025-08-05T23:56:22Z
----------------------------------------------------------------

already did this 2 above


@review-notebook-app
Copy link

review-notebook-app bot commented Aug 5, 2025

View / edit / reply to this conversation on ReviewNB

wildintellect commented on 2025-08-05T23:56:23Z
----------------------------------------------------------------

I'm confused, if you already have a sf dataframe what does this section do?

Sorry I didn't catch this before, but you shouldn't need to make another dataframe once you have an sf object it works with leaflet directly.

https://rstudio.github.io/leaflet/articles/json.html

center <- st_bbox(df_sf) %>% 
    st_as_sfc %>% 
    st_centroid %>%
    st_coordinates

center

# Create the leaflet map centered on the mean coordinates
leaflet_map <- leaflet(df_sf) %>%
  addProviderTiles("OpenStreetMap") %>%
  addCircles(
    radius = 400,
    popup = ~paste("Shot Number:", nrow(df_sf))
  ) %>%
  setView(
      lat = center[2],
      lng = center[1],
      zoom = 12
  )

# Show the map
leaflet_map


@review-notebook-app
Copy link

review-notebook-app bot commented Aug 5, 2025

View / edit / reply to this conversation on ReviewNB

wildintellect commented on 2025-08-05T23:56:23Z
----------------------------------------------------------------

also clean this up to use the df_sf object


@review-notebook-app
Copy link

review-notebook-app bot commented Aug 5, 2025

View / edit / reply to this conversation on ReviewNB

wildintellect commented on 2025-08-05T23:56:24Z
----------------------------------------------------------------

get rid of redundant display_df

duplicating data like this could lead to memory issue with very large data


@review-notebook-app
Copy link

review-notebook-app bot commented Aug 5, 2025

View / edit / reply to this conversation on ReviewNB

wildintellect commented on 2025-08-05T23:56:25Z
----------------------------------------------------------------

also df_sf


@HarshiniGirish
Copy link
Collaborator Author

@wildintellect up for review !

@review-notebook-app
Copy link

review-notebook-app bot commented Aug 11, 2025

View / edit / reply to this conversation on ReviewNB

wildintellect commented on 2025-08-11T23:01:51Z
----------------------------------------------------------------

Line #6.    center <- c(mean(c(bbox["xmin"], bbox["xmax"])), mean(c(bbox["ymin"], bbox["ymax"])))

Line 5 and 6 can be simplified to 1 line , like other st_centroid usage in the notebook.


@review-notebook-app
Copy link

review-notebook-app bot commented Aug 11, 2025

View / edit / reply to this conversation on ReviewNB

wildintellect commented on 2025-08-11T23:01:52Z
----------------------------------------------------------------

Line #11.    center <- st_coordinates(st_centroid(st_union(df_sf)))

Already calculated previously, just pick unique variable names to avoid overwriting


@review-notebook-app
Copy link

review-notebook-app bot commented Aug 11, 2025

View / edit / reply to this conversation on ReviewNB

wildintellect commented on 2025-08-11T23:01:53Z
----------------------------------------------------------------

If you're going to plot world or even regional like the next map, you might as well just plot the centroid, it will plot much faster.

Maybe this map should be regional, with centroid as a single point, and the next map with TMAP should zoom in to the bbox of the the data.


@HarshiniGirish
Copy link
Collaborator Author

@wildintellect thanks! up for review

@review-notebook-app
Copy link

review-notebook-app bot commented Aug 19, 2025

View / edit / reply to this conversation on ReviewNB

wildintellect commented on 2025-08-19T17:45:15Z
----------------------------------------------------------------

[GEDI Subsetter](https://github.com/MAAP-Project/gedi-subsetter/) dataset

Can also link to the MAAP tutorial on using GEDI subsetter https://docs.maap-project.org/en/latest/science/GEDI/GEDI_L4A.html

Parquet -> GeoParquet


@review-notebook-app
Copy link

review-notebook-app bot commented Aug 19, 2025

View / edit / reply to this conversation on ReviewNB

wildintellect commented on 2025-08-19T17:45:16Z
----------------------------------------------------------------

two periods in a row . .


@review-notebook-app
Copy link

review-notebook-app bot commented Aug 19, 2025

View / edit / reply to this conversation on ReviewNB

wildintellect commented on 2025-08-19T17:45:16Z
----------------------------------------------------------------

Line #7.    if (!identical(sf::st_crs(world), crs)) {

the data variable is World not world


@review-notebook-app
Copy link

review-notebook-app bot commented Aug 19, 2025

View / edit / reply to this conversation on ReviewNB

wildintellect commented on 2025-08-19T17:45:17Z
----------------------------------------------------------------

Line #13.    dx <- (bb$xmax - bb$xmin) * 0.15

this is actually too zoomed in *250 works so you can actually tell where this is.


@review-notebook-app
Copy link

review-notebook-app bot commented Aug 19, 2025

View / edit / reply to this conversation on ReviewNB

wildintellect commented on 2025-08-19T17:45:18Z
----------------------------------------------------------------

It's good that this is zoomed in now, but we need to apply colors to the points.

I'm also researching how to add context to the map, but hit a bit a of a problem...

maptiles package is need to pull from online basemaps, but 0.10 requires a newer Terra version

I installed 0.9 with install.packages("https://cran.r-project.org/src/contrib/Archive/maptiles/maptiles_0.9.0.tar.gz", repos=NULL, type="source")

but it still doesn't seem to work.

So I went about trying to find a Raster we could use as background

shaded_relief <- rnaturalearth::ne_download(scale=50, 
                                           type="HYP_50M_SR", 
                                           category="raster")

bb  <- st_bbox(df_sf)
pad <- 25
dx <- (bb["xmax"] - bb["xmin"]) * pad
dy <- (bb["ymax"] - bb["ymin"]) * pad
bb_exp <- structure(
  c(xmin = bb["xmin"] - dx, ymin = bb["ymin"] - dy,
    xmax = bb["xmax"] + dx, ymax = bb["ymax"] + dy),
  class = class(bb), crs = st_crs(df_sf)
)

shaded_relief_crop <- terra::crop(shaded_relief, bb_exp)

tmap_mode("plot")

#tm_shape(world, bbox = bb_exp) +
#  tm_polygons(col = "grey90", border.col = "grey60") + 
tmap::tm_shape(shaded_relief_crop) +
    tmap::tm_rgb(r=3, g=2, b=1) + 
tm_shape(df_sf) +  
    tm_dots(size = 0.1) +
    tm_layout(
    title = "GEDI Footprint Locations (Static Map)",
    frame = FALSE,
    legend.show = FALSE
  )

but this data isn't not high resolution so it also doesn't work well when zoomed in.

So I think this plan can work, we just need to find an easy to get layer to put behind the points, and give the points colors like the above maps.

tmap documentation to reference https://r-tmap.github.io/tmap-book/layers.html


@wildintellect
Copy link
Collaborator

wildintellect commented Aug 21, 2025

It does seem to be a bug that tm_basemap (based on maptiles) doesn't work in Jupyter notebooks. I was able to use direct functions to cache the tiles and plot as a raster.

This will be useful if/when we manage to get updated packages though.

library(maptiles)
base_tiles <- get_tiles(df_sf, zoom=12)
tmap::tm_shape(base_tiles) +
    tmap::tm_rgb(r=3, g=2, b=1) + 
tm_shape(df_sf) +  
    tm_basemap("OpenStreetMap") +
    tm_dots(size = 0.1) +
    tm_layout(
        title = "GEDI Footprint Locations (Static Map)",
        frame = TRUE,
        legend.show = FALSE
  )
image

@HarshiniGirish
Copy link
Collaborator Author

HarshiniGirish commented Sep 2, 2025

@omshinde @smk0033 the updates to the last static plot has been made. Since Alex is out please review this notebook at your convenience. Thank you

@HarshiniGirish
Copy link
Collaborator Author

@wildintellect up for review!

@wildintellect
Copy link
Collaborator

View / edit / reply to this conversation on ReviewNB

wildintellect commented on 2025-08-19T17:45:17Z ----------------------------------------------------------------

Line #13. dx <- (bb$xmax - bb$xmin) * 0.15

this is actually too zoomed in *250 works so you can actually tell where this is.

250 not 0.25

the map should look like
image

Make that change and we'll call version 1 of this tutorial done.
Also @smk0033 this is in the visualization folder. Should we move it to the working_with_r folder for consistency?

@wildintellect
Copy link
Collaborator

@HarshiniGirish Looks like you need to resolve a conflict now before we can approve.
Also reminder, please stop using git rm so often, moving the file should have been a git mv

@HarshiniGirish HarshiniGirish deleted the Vector-data branch September 10, 2025 16:50
@wildintellect wildintellect mentioned this pull request Sep 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants