defense v0.5.1
Defense
🔮 A Crystal HTTP handler for throttling, blocking and tracking malicious requests 🔮
Getting started
Installation
Add the shard as a dependency to your project's shards.yml
:
dependencies:
defense:
github: defense-cr/defense
...and install it:
shards install
Configure the data store
Defense stores its state in a Redis database. You can configure this by setting the REDIS_URL
environment variable or by using the Defense#store=
method:
Defense.store = Defense::RedisStore.new(url: "redis://localhost:6379/0")
For simple use cases or tests you can also use the memory store:
Defense.store = Defense::MemoryStore.new
You can always implement your own custom store by extending the abstract class Defense::Store
.
Plugging into the application
Defense is built as a Crystal HTTP::Handler
. You will need to register the Defense::Handler
to your web application's handler chain. For more information about handlers and the handler chain follow this link.
Usually the earlier you register the handler to your handler chain, the better. This ensures that malicious requests are blocked early own, before other layers (handlers) of your application are reached.
Here's how to plug Defense into some of the most popular Crystal web frameworks:
Kemal
In Kemal you would use the add_handler
method to register the Defense handler:
require "kemal"
require "defense"
add_handler Defense::Handler.new
# Other handlers...
add_handler SomeOtherHandler.new
get "/" do
"hello world"
end
Kemal.run
For more details, check out the kemal-defense-example repository.
Amber
In Amber you register handlers as part of a pipeline in your config/routes.cr
file:
Amber::Server.configure do |app|
pipeline :web do
plug Defense::Handler.new
# Other handlers...
plug SomeOtherHandler.new
end
routes :web do
get "/", HomeController, :index
end
end
Lucky
In Lucky, you would add the Defense::Handler
within your src/app_server.cr
file, somewhere before the Lucky::RouteHandler
:
class AppServer < Lucky::BaseAppServer
def middleware
[
Defense::Handler.new,
# Other handlers...
SomeOtherHandler.new,
Lucky::RouteHandler.new,
]
end
end
HTTP::Server (Standalone)
When using the standard library HTTP::Server
, any middleware is registered as part of the initializer:
require "defense"
require "http/server"
server = HTTP::Server.new([Defense::Handler.new]) do |context|
context.response.content_type = "text/plain"
context.response.print "hello world"
end
server.bind_tcp(8080)
server.listen
Usage
Defense provides a set of configurable rules that you can use to throttle, block and track malicious requests based on your own heuristics:
- Throttling
- Configure the throttled response
- Blocklist
- Configure the blocked response
- Fail2Ban
- Allow2Ban
- Safelist
Throttling
The Defense.throttle
method can be used to throttle clients based on a maximum number of requests (limit) over a given time frame specified in seconds (period).
The method takes a block which receives the request
as an argument. The return value of the block should either be nil
(in which case the request will not be counted at all) or a String
which uniquely identifies the client to throttle. A good identifier is usually the IP address.
The following example throttles clients based on their IP address to a limit of 10 requests per minute:
Defense.throttle("throttle requests per minute", limit: 10, period: 60) do |request|
request.remote_address.to_s
end
The following example throttles clients in a similar way but will ignore requests coming from 127.0.0.1
:
Defense.throttle("throttle requests per minute except localhost", limit: 10, period: 60) do |request|
return nil if request.remote_address.to_s == "127.0.0.1"
request.remote_address.to_s
end
Configure the throttled response
Throttled requests are responded with:
HTTP/1.1 429 Too Many Requests
content-type: text/plain
content-length: 10
Retry later
You can override the default response message by using the Defense.throttled_response=
method:
Defense.throttled_response = ->(response : HTTP::Server::Response) do
response.status = HTTP::Status::UNAUTHORIZED
response.content_type = "application/json"
response.puts("{'hello':'world'}")
end
Blocklist
The Defense.blocklist
method can be used to block malicious or unwanted requests.
The method takes a block which receives the request
as an argument. The return value of the block should either be true
- in which case the request will be blocked, or false
- in which case the request will be allowed.
The following example blocks all requests to /admin/*
:
Defense.blocklist("block requests to the admin") do |request|
request.path.starts_with?("/admin/")
end
The following example blocks requests based on a predefined list of malicious IPs:
MALICIOUS_IPS = ["1.1.1.1", "2.2.2.2", "3.3.3.3"]
Defense.blocklist("block requests from malicious ips") do |request|
MALICIOUS_IPS.includes?(request.remote_address.to_s)
end
The Spamhaus DROP lists are a great resource for malicious IPs to block.
Configure the blocked response
Blocked requests are responded with:
HTTP/1.1 403 Forbidden
content-type: text/plain
content-length: 9
Forbidden
You can override the default response message by using the Defense.blocked_response=
method:
Defense.blocked_response = ->(response : HTTP::Server::Response) do
response.status = HTTP::Status::UNAUTHORIZED
response.content_type = "application/json"
response.puts("{'hello':'world'}")
end
Fail2Ban
The Defense::Fail2Ban.filter
method can be used within a Defense.blocklist
block to ban misbehaving clients for a given period of time (bantime) after a sequence of blocked requests (maxretry) performed over a particular time range (findtime).
The method's first argument should be a unique identifier of the client - the IP address is usually a safe bet. It's highly recommended to namespace this identifier, in order to avoid conflicts with other Fail2Ban
or Allow2Ban
calls - e.g. my-fancy-filter:#{request.remote_address.to_s}
would be a good identifier.
The method also takes a block which should return true
- in which case the request will be blocked and counted for the ban, or false
- in which case the request will be allowed and excluded from the ban count. Note that the return value of the #filter
block will also be used as a return value for the #blocklist
block.
The following example blocks any requests containing /etc/passwd
inside the path and, once a particular client identified by IP has accumulated 5 requests wihin 60 seconds, it bans him for the next 24 hours:
Defense.blocklist("fail2ban pentesters") do |request|
Defense::Fail2Ban.filter("pentesters:#{request.remote_address.to_s}", maxretry: 5, findtime: 60, bantime: 24 * 60 * 60) do
request.path.includes?("/etc/passwd")
end
end
Allow2Ban
The Defense::Allow2Ban.filter
method works the same way as Defense::Fail2Ban.filter
except that it allows requests from misbehaving clients until such time as they reach maxretry at which they are cut off as per normal.
The following example allows all POST /login
requests until a particular client identified by IP has accumulated 5 requests within 60 seconds, at which point it bans him for the next 24 hours:
Defense.blocklist("allow2ban too many login attempts") do |request|
Defense::Allow2Ban.filter("too-many-login-attempts:#{request.remote_address.to_s}", maxretry: 5, findtime: 60, bantime: 24 * 60 * 60) do
request.method == "POST" && request.path == "/login"
end
end
Safelist
The Defense.safelist
method can be used to exclude requests from any throttling or blocking rules. This method has precedence over all the other rules.
The method takes a block which receives the request
as an argument. The return value of the block should either be true
- in which case the request will never be throttled or blocked, or false
- in which case the request will be checked against the other existing rules and might potentially be throttled or blocked.
The following example marks all requests originating from 127.0.0.1
as safe:
Defense.safelist("local requests are safe") do |request|
request.remote_address.to_s == "127.0.0.1"
end
Contributing & Development
Contributions are welcome. Make sure to check the existing issues (including the closed ones) before requesting a feature, reporting a bug or opening a pull requests.
Getting started
Install dependencies:
shards install
Run tests using Redis as a backend (requires a running Redis server):
crystal spec
Run tests using the memory store as a backend:
STORE=memory crystal spec
Format the code:
crystal tool format
Guidelines
- Keep the public interface small. Anything that doesn't have to be public, should explicitly be marked as protected or private, including classes.
- Be explicit about type declaration (especially on public methods).
- Use the Crystal formatter to format the code.
- For now, prefer integration/system tests over unit tests.
Maintainers
Credits
This shard is heavily inspired by rack-attack ❤
defense
- 60
- 3
- 0
- 3
- 2
- 10 months ago
- May 17, 2019
Wed, 20 Nov 2024 23:00:18 GMT