Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion http/get_range/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,4 @@

# HTTP GET Arrow Data: Range Request Examples

This directory contains examples of HTTP servers/clients that send/receive data of known size (`Content-Length`) in the Arrow IPC streaming format and support range requests (`Accept-Range: bytes`).
This directory contains examples of HTTP servers/clients that send/receive data of known size (`Content-Length`) in the Arrow IPC streaming format and support range requests (`Accept-Ranges: bytes`).
19 changes: 19 additions & 0 deletions http/get_range/curl/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

*.arrows
*.arrows.part*
24 changes: 24 additions & 0 deletions http/get_range/curl/client/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
<!---
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# HTTP GET Arrow Data: Range Request curl Client Example

This directory contains examples of `curl` commands that send HTTP GET requests with the `Range` request header.

To run this example, first start one of the range request server examples in the parent directory, then run the shell commands in `client.sh`.
50 changes: 50 additions & 0 deletions http/get_range/curl/client/client.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
#!/bin/sh

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.


### Use range requests to download an Arrow IPC stream file in two parts

# Get the length of the file `random.arrows` in bytes
curl -I localhost:8008/random.arrows
# Content-Length: 13550776

# Download the first half of the file to `random.arrows.part1`
curl -r 0-6775388 localhost:8008/random.arrows -o random.arrows.part1

# Download the second half of the file to `random.arrows.part2`
curl -r 6775389-13550776 localhost:8008/random.arrows -o random.arrows.part2

# Combine the two separate files into one file `random.arrows` then delete them
cat random.arrows.part1 random.arrows.part2 > random.arrows
rm random.arrows.part1 random.arrows.part2

# Clean up
rm random.arrows


### Simulate an interrupted download over a slow connection

# Begin downloading the file at 1M/s but interrupt after five seconds
timeout 5s curl --limit-rate 1M localhost:8008/random.arrows -o random.arrows

# Resume the download at 1M/s
curl -C - --limit-rate 1M localhost:8008/random.arrows -o random.arrows

# Clean up
rm random.arrows
21 changes: 21 additions & 0 deletions http/get_range/js/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

/**/node_modules
package-lock.json
package.json
*.arrows
37 changes: 37 additions & 0 deletions http/get_range/js/server/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
<!---
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# HTTP GET Arrow Data: Range Request Node.js Server Example

The example in this directory shows how to use the Node.js package [`serve`](https://www.npmjs.com/package/serve) (which supports range requests) to serve a static Arrow IPC stream file over HTTP.

To run this example, copy the file `random.arrows` from the directory `data/rand-many-types/` into the current directory:

```sh
cp ../../../../data/rand-many-types/random.arrows .
```

Then start the HTTP server to serve this file:

```sh
npx serve -l 8008
```

> [!NOTE]
> The npm package `serve` _should_ automatically set the `Content-Type` header to `application/vnd.apache.arrow.stream` when serving a file with extension `.arrows`, because [the Arrow IPC stream format is officially registered with IANA](https://www.iana.org/assignments/media-types/application/vnd.apache.arrow.stream) and most web servers including `serve` use registration data from IANA to determine the media type of a file based on its file extension and set the `Content-Type` header to that media type when serving a file with that extension. However, this is not working with `.arrows` files in the `serve` package, seemingly because of a problem with the npm package [`mimedb`](https://github.com/jshttp/mime-db) which `serve` depends on. So the file `serve.json` is used to set the `Content-Type` header correctly when serving `.arrows` files.
11 changes: 11 additions & 0 deletions http/get_range/js/server/serve.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"headers": [
{
"source" : "**/*.arrows",
"headers" : [{
"key" : "Content-Type",
"value" : "application/vnd.apache.arrow.stream"
}]
}
]
}