feat: add v1 spec

[aheev]

added v1 specs for memory and disk

[aheev]

@adsharma could you PTAL?

[adsharma]

create the graph in the database

Creates metadadata. Data stays in parquet files. Could be remote (in object storage)

[adsharma]

format could be optional. Could be inferred from the other parameter

[adsharma]

a URL could be more appropriate here, since it could be a s3://... like URL and anything supported by VFS. Would love to see XET here as well, since some of our datasets live on huggingface.

This is currently called STORAGE. There is some cost to renaming it - so I'd leave it alone if its not the most important thing.

[adsharma]

Also suggest merging the two into a single url. Previous comment about STORAGE applies here as well.

[aheev]

@adsharma can we rid of init script and metadata file from the spec? Because these are completely impl specific. We are not really specifying anything here

[aheev]

Re: comments on examples

@adsharma those are just examples to show the tables / metadata might look, not really an impl of the spec. I will create a baseline impl, which would be based on ladybugDB impl, in a future PR

[adsharma]

can we rid of init script

This whole spec is modeled after DuckLake. Specification: https://ducklake.select/docs/stable/specification/introduction

DuckLake : SQL :: Icebug-Disk : Cypher

If we're going to spend some time writing specs, it makes sense to specify what data types are valid in schema.cypher.

[adsharma]

I will create a baseline impl, which would be based on ladybugDB impl, in a future PR

It's not clear that another impl needs to live in this repo. If lets say Grafeo or ArcadeDB want to implement this spec, the impl could live in their respective repos.

[aheev]

I will create a baseline impl, which would be based on ladybugDB impl, in a future PR

It's not clear that another impl needs to live in this repo. If lets say Grafeo or ArcadeDB want to implement this spec, the impl could live in their respective repos.

Initially, I had the same idea, but I was concerned about the helper scripts like icebug-format.py. If you’re planning to remove them, I'm okay with having impls in their own repos

[aheev]

can we rid of init script

This whole spec is modeled after DuckLake. Specification: https://ducklake.select/docs/stable/specification/introduction

DuckLake : SQL :: Icebug-Disk : Cypher

If we're going to spend some time writing specs, it makes sense to specify what data types are valid in schema.cypher.

isn't icebug-format about graph storage rather than full-fledged graph database 🤔 . How queries are executed is part of query engine right?

[adsharma]

isn't icebug-format about graph storage rather than full-fledged graph database

curl -LO https://datasets.ldbcouncil.org/snb-interactive-v1-ducklake/sf1.ducklake
duckdb
memory D ATTACH 'ducklake:sf1.ducklake' (AUTOMATIC_MIGRATION TRUE);
memory D .schema

Now sf1.ducklake contains only the schema. This is fundamental. Without this, there is no difference between icebug and graphar other than minor syntax.

I imagine a similar sf1.icebug after the spec is fully implemented. It'd be a valid lbdb database that you can open with lbug cli. Contains only the schema and no data.

[aheev]

isn't icebug-format about graph storage rather than full-fledged graph database

curl -LO https://datasets.ldbcouncil.org/snb-interactive-v1-ducklake/sf1.ducklake
duckdb
memory D ATTACH 'ducklake:sf1.ducklake' (AUTOMATIC_MIGRATION TRUE);
memory D .schema

Now sf1.ducklake contains only the schema. This is fundamental. Without this, there is no difference between icebug and graphar other than minor syntax.

I imagine a similar sf1.icebug after the spec is fully implemented. It'd be a valid lbdb database that you can open with lbug cli. Contains only the schema and no data.

got it now. Changes in the latest commit