HttpFS Gateway

Ozone HttpFS can be used to integrate Ozone with other tools via REST API.

Introduction

Ozone HttpFS is forked from the HDFS HttpFS endpoint implementation (HDDS-5448). Ozone HttpFS is intended to be added optionally as a role in an Ozone cluster, similar to S3 Gateway.

HttpFS is a service that provides a REST HTTP gateway supporting File System operations (read and write). It is interoperable with the webhdfs REST HTTP API.

HttpFS can be used to access data on an Ozone cluster behind of a firewall. For example, the HttpFS service acts as a gateway and is the only system that is allowed to cross the firewall into the cluster.

HttpFS can be used to access data in Ozone using HTTP utilities (such as curl and wget) and HTTP libraries Perl from other languages than Java.

The webhdfs client FileSystem implementation can be used to access HttpFS using the Ozone filesystem command line tool (ozone fs) as well as from Java applications using the Hadoop FileSystem Java API.

HttpFS has built-in security supporting Hadoop pseudo authentication and Kerberos SPNEGO and other pluggable authentication mechanisms. It also provides Hadoop proxy user support.

Getting started

HttpFS service itself is a Jetty based web-application that uses the Hadoop FileSystem API to talk to the cluster, it is a separate service which provides access to Ozone via a REST APIs. It should be started in addition to other regular Ozone components.

To try it out, follow the instructions from the link below to start the Ozone cluster with Docker Compose.

https://ozone.apache.org/docs/edge/start/startfromdockerhub.html

docker compose up -d --scale datanode=3

You can/should find now the HttpFS gateway in docker with the name like ozone_httpfs, and it can be accessed through localhost:14000. HttpFS HTTP web-service API calls are HTTP REST calls that map to an Ozone file system operation.

Here’s some example usage:

Create a volume

# creates a volume called `volume1`.
curl -i -X PUT "http://localhost:14000/webhdfs/v1/volume1?op=MKDIRS&user.name=hdfs"

Example Output:

HTTP/1.1 200 OK
Date: Sat, 18 Oct 2025 07:51:21 GMT
Cache-Control: no-cache
Expires: Sat, 18 Oct 2025 07:51:21 GMT
Pragma: no-cache
Content-Type: application/json
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Set-Cookie: hadoop.auth="u=hdfs&p=hdfs&t=simple-dt&e=1760809881100&s=OCdVOi8eyMguFySkmEJxm5EkRfj6NbAM9agi5Gue1Iw="; Path=/; HttpOnly
Content-Length: 17

{"boolean":true}

Create a bucket

# creates a bucket called `bucket1`.
curl -i -X PUT "http://localhost:14000/webhdfs/v1/volume1/bucket1?op=MKDIRS&user.name=hdfs"

Example Output:

HTTP/1.1 200 OK
Date: Sat, 18 Oct 2025 07:52:06 GMT
Cache-Control: no-cache
Expires: Sat, 18 Oct 2025 07:52:06 GMT
Pragma: no-cache
Content-Type: application/json
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Set-Cookie: hadoop.auth="u=hdfs&p=hdfs&t=simple-dt&e=1760809926682&s=yvOaeaRCVJZ+z+nZQ/rM/Y01pzEmS9Pe2mE9f0b+TWw="; Path=/; HttpOnly
Content-Length: 17

{"boolean":true}

Upload a file

echo "hello" >> ./README.txt
curl -i -X PUT "http://localhost:14000/webhdfs/v1/volume1/bucket1/user/foo/README.txt?op=CREATE&data=true&user.name=hdfs" -T ./README.txt -H "Content-Type: application/octet-stream" 

Example Output:

HTTP/1.1 100 Continue

HTTP/1.1 201 Created
Date: Sat, 18 Oct 2025 08:33:33 GMT
Cache-Control: no-cache
Expires: Sat, 18 Oct 2025 08:33:33 GMT
Pragma: no-cache
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Set-Cookie: hadoop.auth="u=hdfs&p=hdfs&t=simple-dt&e=1760812413286&s=09t7xKu/p/fjCJiQNL3bvW/Q7mTw28IbeNqDGlslZ6w="; Path=/; HttpOnly
Location: http://localhost:14000/webhdfs/v1/volume1/bucket1/user/foo/README.txt
Content-Type: application/json
Content-Length: 84

{"Location":"http://localhost:14000/webhdfs/v1/volume1/bucket1/user/foo/README.txt"}

Read the file content

# returns the content of the key `/user/foo/README.txt`.
curl 'http://localhost:14000/webhdfs/v1/volume1/bucket1/user/foo/README.txt?op=OPEN&user.name=foo'
hello

Supported operations

Here are the tables of WebHDFS REST APIs and their state of support in Ozone.

File and Directory Operations

Other File System Operations

Hadoop user and developer documentation about HttpFS

• HttpFS Server Setup

• Using HTTP Tools