Using hpfriends - the social data-sharing platform

11 May 2013

Abstract: This post introduces the hpfriends system, which is a a new sharing model and web frontend for the hpfeeds data-sharing platform. Readers will learn how to use the hpfriends frontend to publish and subscribe to data-channels in the hpfeeds system and share their data with other users. There is a separate post on the architecture of hpfriends.

TOC: Using hpfriends, using hpfeeds and using hpfeeds with dionaea.

Introduction

hpfeeds is a simple publish/subscribe data sharing model. It was initially created by Mark ‘rep’ Schloesser as a way to carry high-volume real-time data from different pieces of honeypot software between members of the Honeynet project. hpfriends is an evolution of the hpfeeds data sharing model. It uses the same wire protocol and thus maintains backwards-compatibility with all data sources / sinks.

However, instead of relying on access authorization based on channels, hpfriends uses a social graph in order to make sharing data even easier and more natural. This also circumvents some policy issues about which people should be able to use hpfeeds and who should be allowed onto which channels.

Using hpfriends  

This is the main menu of hpfriends. It will appear on every page.

Shares display the channels you share with other users and group. Shared with you is data that other users share with you, either directly or via groups.

You can create and edit permissions for your authkeys under Authkeys, while Logs contain information about failed attempts to use your authkeys.

If you click Groups, you’ll be able to create groups and see your memberships in other groups. The Search page allows you to quickly search for channels, log messages and groups.

Sharing data

This view shows channel names that you currently share with other users and groups. Click Add share to define an additional sharing relationship.

Currently shared channels.

The Type field specifies whether you only want to share data generated by yourself for that channel name or if you also want to redistribute data on that channel which you received from other users. Use the search-field above the select menu for users to filter for usernames.

Adding a new share.

Accepting data

If other users share data channels with you, these shares will appear under Shared with you.

Data shared by other users.

This view shows that there are two channels currently shared with me directly. I have activated the channel mobile.malware and could now receive and forward messages by this user on this channel. Since I am not interested in receiving data by user leacab on channel fooo, I didn’t activate it.

Authkeys

If you want to publish and subscribe to data channels you have to add authkeys. The Authkeys view shows one of my authkeys. It has different permission for publishing and subscribing, along with statistics about messages and bytes sent and received. Authkeys should be used as single-purpose tokens for different pieces of backend software, with the smallest necessary amount of privileges. This will reduce the impact if one of your sensors is compromised.

List and statistics of authkeys.

Groups

Groups can be used to conveniently group and subsequently share data with a number of people. Currently, the group-functionality is somewhat limited. It is not possible to leave groups or to remove members.

Logs

The logs will contain error messages which arise when you misuse your authkeys. Check the logs first if you’re having problems publishing and subscribing to a channel.

Log messages indicate possible error sources with authkeys.

Using hpfeeds  

At the end of the day, you want to use hpfeeds to publish your data and subscribe to data channels. This is where the hpfeeds cli tools come in. This section will show you how to use the tools in combination with hpfriends.

hpfeeds - python implementation

The “reference” hpfeeds implementation was created by Mark and should be used in most cases. Simply get the source code from GitHub, no setup necessary.

There is small tool which you can use to manually publish and subscribe to channels. It is located in the cli folder and can be used as follows:

hpfeeds-client -i ident -s secret --host host -p port -c channel1 [-c channel2, ...] 'action' [<data>]

Where ‘action’ can be publish or subscribe.

Looking at our list of authkeys above, we could use our key to publish to channel random.int:

hpfeeds-client --host hpfriends.honeycloud.net -p 20000 -i zNKeYNK -s KwxHqvJ -c random.int publish "42"

We could also subscribe to one of the channels which our authkey has permissions to:

hpfeeds-client --host hpfriends.honeycloud.net -p 20000 -i zNKeYNK -s KwxHqvJ -c geoloc.response subscribe

Using hpfeeds with dionaea  

One of the sensors that supports hpfeeds is dionaea - the most modern low-interaction server honeypot. This software is also the source for most events currently showing up on the HoneyMap. Sadly the hpfeeds module is not part of the official git repository, so we need to make a small change to its setup steps.

To setup dionaea you need to follow the official setup instructions (dionaea.carnivore.it) and then use the git URL git://github.com/rep/dionaea.git instead of the normal dionaea one. This will include the hpfeeds module and the appropriate config section in the honeypot. After it compiled correctly and you installed dionaea on your system, you need to edit the dionaea.conf file and enter your hpfeeds credentials, as well as enable the module. You get the authkey credentials by creating some in the hpfriends webinterface. The following is how the needed config sections look like:

dionaea.conf

     hpfeeds = {
       hp1 = {
         server = "hpfriends.honeycloud.net"
         port = "10000"
         ident = "<your_authkey_identifier>"
         secret = "<your_authkey_secret>"
         // dynip_resolve: enable to lookup the sensor ip through a webservice
         dynip_resolve = "https://hpfriends.honeycloud.net/ip"
       }
     }
...
     ihandlers = {
...
//    "submit_http",
      "hpfeeds",
//    "logxmpp",
...

Remember to also give your authkey the necessary access rights. It needs to be able to publish on the channels dionaea.capture, dionaea.shellcodeprofiles, dionaea.dcerpcrequests and mwbinary.dionaea.sensorunique.

After this is done you can start the honeypot and your captured attacks should be appearing on the hpfriends “dionaea.capture” channel. If you share this channel with Mark or the Honeynet Project in general and tell Mark about it, then he can also make your captured attacks on the HoneyMap with one click in the hpfriends interface.

Further reading

hpfriends is maintained by Mark ‘rep’ Schloesser and Johannes ‘heipei’ Gilger

• hpfriends - the hpfriends website
• hpfeeds - hpfeeds GitHub repo
• hpfeeds-ruby - Ruby gem for hpfeeds
• go-hpfeeds - Go implementation of hpfeeds
• Honeynet - The Honeynet Project
• HoneyMap - The HoneyMap, which uses hpfeeds