rVM going public

Okay, here we go, I published some ruby gems with rVM, it is way not done and leaks some important features but it is working already. The latest version, SVN trunk, currently supports a quite nice and complete math interpreter. Sure it does not do as much as some full blown implementations yet it can do quite a lot.

It supports now:

• All usuall opperations, +, *, -, /, ^ with right priorities.
• Functions sin(123), and custom functions if you wish.
• Parentheses to group operations and overwrite operator priorities.
• Variables, as in x + 3. Those can be set outside of the code by manipulating the passed environment.
• Assignments, as in x=23+19. Which can be read outside the code by reading from the passed environment.
• Sequences of expressions separated by the ; operator.

With the latest two items here new to the implementation and perhaps most important. And most interesting. One feature added today is allowing to generate sequences by using ; as a separator, thus making it possible to chain up operations over multiple lines.

If you feel like checking it out:

svn co http://code.licenser.net/rvm/trunk rvm cd rvm rake gem gem install pkg/*

also

rdoc README lib/

gives you a nice overview about some things as well as the short howto and example at

http://code.licenser.net/documents/show/5

That will install you the latest gem from the trunk – Have fun and I’m always happy about feedback ^^ and or tickets at:

http://code.licenser.net/projects/show/rvm

I think there are about one million trillion benchmarks out there that compare ruby 1.8 with ruby 1.9. And about any one of them has a different judgement on things. So I figured hey I’m going to give it a look too.

First of all, what I’m looking at? Nope not the speed of how things execute today, but what it execute. It has to do with some nifty tool I found called SytemTap it offers some utilities to trace what a application is doing. For example, on what I’m looking here, what system calls it performs. This can be quite nifty to see where bottlenecks are.

Before we go into some details lets first think about what that means for the results. They are completely independent from the environment, that is good, meaning a second process running along side won’t influence the results. Then again at times the meaning of them can be quite questionable, admitted I have no clue what about 90% of them even do. A third point is that even a process with no system calls at all can be slow as hell.

Non the less there are some things that, at least for me have a quite clear meaning. so lets take a look at some simple examples.

I took the bm_app_mandelbrot.rb file distributed with ruby 19, jus because well I liked it, I know it’s not the most valid example but it’s output is nice and short and the results had been the same with about any other test I ran.

Ruby 1.9:

sys_munmap              calls:     4    avg time (us):   27     total(us):    109
sys_mprotect            calls:    12    avg time (us):   21     total(us):    257
sys_mmap                calls:    39    avg time (us):    7     total(us):    297
sys_rt_sigaction        calls:    32    avg time (us):    5     total(us):    178
sys_rt_sigprocmask      calls:    11    avg time (us):    5     total(us):     57
sys_read                calls:    19    avg time (us):   36     total(us):    690
sys_rt_sigtimedwait     calls:     1    avg time (us):    7     total(us):      7
sys_brk                 calls:    17    avg time (us):    6     total(us):    105
sys_open                calls:    18    avg time (us):   15     total(us):    276
sys_getcwd              calls:     1    avg time (us):    8     total(us):      8
sys_getdents64          calls:     2    avg time (us):   69     total(us):    139
sys_getrlimit           calls:     6    avg time (us):    5     total(us):     32
sys_getpgrp             calls:     1    avg time (us):    5     total(us):      5
sys_getppid             calls:     1    avg time (us):    5     total(us):      5
sys_getpid              calls:     1    avg time (us):    5     total(us):      5
sys_getegid             calls:     3    avg time (us):    4     total(us):     14
sys_geteuid             calls:     5    avg time (us):    5     total(us):     25
sys_getgid              calls:     2    avg time (us):    5     total(us):     10
sys_getuid              calls:     2    avg time (us):    5     total(us):     11
sys_lseek               calls:     3    avg time (us):    5     total(us):     16
sys_ioctl               calls:     5    avg time (us):    7     total(us):     38
sys_arch_prctl          calls:     2    avg time (us):    5     total(us):     10
sys_newstat             calls:    15    avg time (us):   46     total(us):    698
sys_newfstat            calls:    15    avg time (us):    5     total(us):     80
sys_futex               calls:     1    avg time (us):13408     total(us):  13408
sys_fcntl               calls:     1    avg time (us):    5     total(us):      5
sys_faccessat           calls:    13    avg time (us):    6     total(us):     86
sys_select              calls:     5    avg time (us):    7     total(us):     36
sys_set_tid_address     calls:     1    avg time (us):    5     total(us):      5
sys_clone               calls:     1    avg time (us):   19     total(us):     19
sys_close               calls:    21    avg time (us):    5     total(us):    125
sys_uname               calls:     3    avg time (us):    5     total(us):     17

Lets have a look, there are a hell of a number of words that are hard to understand, but lets just focus on 2 things,

All calls are pretty much even, in the number of calls, okay even with a difference of 38 calls. It is not per se a good thing but once we look ahead we’ll see that, at least it’s not a bad thing.

The second interesting thing is that the call sys_futex took the most time, which makes sense as it is a method used to wait for a memory access so I’d say it’s nothing unusual.

Now it gets interesting, lets have a look at ruby 1.8:

sys_munmap              calls:     6    avg time (us):   30     total(us):    182
sys_mprotect            calls:    11    avg time (us):   20     total(us):    221
sys_mmap                calls:    40    avg time (us):    7     total(us):    295
sys_rt_sigaction        calls:    30    avg time (us):    5     total(us):    151
sys_rt_sigprocmask      calls:2022790   avg time (us):    5     total(us):10252567
sys_read                calls:    20    avg time (us):   11     total(us):    227
sys_brk                 calls:    22    avg time (us):    6     total(us):    148
sys_open                calls:    21    avg time (us):   10     total(us):    222
sys_getrlimit           calls:     2    avg time (us):    5     total(us):     10
sys_getpgrp             calls:     1    avg time (us):    5     total(us):      5
sys_getppid             calls:     1    avg time (us):    5     total(us):      5
sys_getpid              calls:     1    avg time (us):    5     total(us):      5
sys_getegid             calls:     3    avg time (us):    4     total(us):     14
sys_geteuid             calls:     3    avg time (us):    5     total(us):     15
sys_getgid              calls:     2    avg time (us):    5     total(us):     10
sys_getuid              calls:     2    avg time (us):    5     total(us):     10
sys_lseek               calls:     2    avg time (us):    5     total(us):     10
sys_arch_prctl          calls:     2    avg time (us):    5     total(us):     11
sys_newstat             calls:    21    avg time (us):   11     total(us):    249
sys_newfstat            calls:    15    avg time (us):    5     total(us):     81
sys_faccessat           calls:    13    avg time (us):    6     total(us):     87
sys_set_tid_address     calls:     1    avg time (us):    5     total(us):      5
sys_close               calls:    24    avg time (us):    5     total(us):    136
sys_uname               calls:     3    avg time (us):    5     total(us):     16

Okay here sys_rt_sigprocmask is most interesting, alone cause it is called like 2 million times o.O I think we can agree that there is a difference to ruby 1.9 even without exactly knowing what the heck it is doing. So I did a bit of research cause that kind of jumped in my attention, it sets some kind of flags for the process.

Doesn’t sound so bad does it? Now you’ll love to hear this: about most of them do exactly the same, and when I understood it correctly the call should not do anything in that case.

A little quote:

rt_sigprocmask changes the list of currently blocked signals. The set value stores the signal mask of the pending signals. The previous ac- tion on the signal is saved in oact. The value of how indicates how the call should behave; its values are as follows: SIG_BLOCK The set of blocked signals is the union of the current set and the set argument.

Now I modified the SystemTap script a bit to show me what kind of sys_rt_sigprocmask calls are made and it reveals that the majority of them use SIG_BLOCK and a null pointer for the new set. Now when I understand this call correct this is a completely useless call to remember the man page, ‘the set of blocked signals is a union’ and some basic math one might get the feeling ‘a union with an empty set is the set itself’

What does that say? Honesty in full extend that is pretty much open to interpretation. If you’d ask me, it says that there is no question a huge improvement in the code from 1.8 to 1.9.

If someone has a nice long script that runs out of the box and is not some kind of serve that will run forever, please let me know. I’m still looking for other tests on this.

Okay,
Here I found a new peace of software. A lot of you will know Trac already, it’s a nice tool. So recently I got hinted to redmine. It is also a issue tracker, and seems to be quite similar in it’s functionality to Trac. Good part is, it’s written in ruby.

The for me most interesting feature is that you can have multiple projects at once. Beside of that it of cause has a Wiki, bug reports, feature requests, version management, a repository browse and a bunch of more features.

Now the best news is that I’ll not use this thing with nothing, two projects (RMush and RedICE) are right now there, including read access to the SVN repository. Of cause I hope that some people are going to try them out ;) and give some feedback. Bugs are fixed quicker when reported – I promise!

Lice! – code

Lice.

Okay just a short note here.

Ruby Mush goes public, kind of at least. I’Ve set up a issue tracker for people who are interested to follow the development.

The current task I am at is writing specs which is a nice alternative to Test::Unit and at least for me, way better suited.

So, there we go. I started a new project and so write a new article here.

It’s something interesting again, and something on the same tracks as before in some ways.
I love MUSH’S, and I somewhat dislike the language used with them (known as MUSHCode).

I tried to work around this by making me a nice IDE to work with the code but honestly I didn’t had much luck with that. It worked but it didn’t really helped as much as I hoped.

Well what to do? I figured if I can’t make writing MUSHCode less unpleasant I just find a way to minimize the need for it. So I started to write a own MUSH server. Of cause in Ruby. I plan it to be highly modular, and based (beside a very basic core) based on plugins.

Don’t get me wrong, there is a place for MUSHCode, it is the best language I ever found to do the so very important ANSI/ASCII interfaces that make MUSH’s what they are. So there will of cause be the option to use MUSHCode.

Actually I went a step further and made languages a plugin. Sounds nice doesn’t it? But more interesting is what it means. But later more to that.

General Layout

Lets start out with what Ruby MUSH is designed. It features 3 layers of code. Each layer has it’s own reason for existence and it’s separated tasks to handle.

The Core (layer 1)

From first, the core code that is very generalized to make sure the upper layers are not restricted by it. The Core itself handles the whole network mess, aka connecting players, keeping track of connections, closing them, database handling.

Also and perhaps most important it gives the interface for the second layer, it manages the plugins and gives them the interfaces and access to all the data they need to do their jobs. Beyond that it doesn’t do anything, no commands, no functions, no languages.

The Plugins (layer 2)

The second layer are the plugins. They are written in ruby, and on the server so they are pretty close to the core due to that, they have full access to all data they need.

I will go into details about the different types of plugins in a bit but first some general things. There will ship plugins with the Ruby MUSH core, those plugins are either essential for the server to work or I like them so much that I included them and think they are worthwile for every place to have around.

People may now ask, why won’t I want to include all plugins in the server that one can find? There are three answers:

There may be conflicts between two plugins. Coder a) might have written their supper cool board plugins and coder b) might have written a plugin too that handles boards both use the same commands → Kaboom.

Every plugin may have a security hole, the more plugins that are loaded the more possible security risks are there so if you know you won’t need space code in your medieval themed MUSH don’t use a space code plugin.

Finally each plugin has a memory footprint and as an impact on speed – so keep plugins out that you don’t need it will keep your server faster.

Command Plugins

Command pluins are plugins that offer commands for the users of the MUSH, an example here are the login commands that are commands offered to the user prior to logging in. A board system would be a command plugin, a chargen would be a command plugin …

Function Plugins

Function plugins are plugins that bring functions, they are either used in the third layer or within the other plugins they are automatically offered within other plugin code. An example for a simple function would be math functions as sqrt that shall be usable in the 3rd layer for softcoding.

Class Plugins

Class plugins are data types used mostly withinthe second layer, the List class is a class plugin, as is the string, an error, a player, a thing, a exit and so on.

Language Plugins

This is perhaps the most interesting part, layer three languages are nothing but plugins. They are mainly compilers that compile the code into a meta lange that can be interpreted by the VM that the layer one brings along.

The big advantage about this is a) you can have any language you wish to as a layer three language. Currently a MUSHCode compiler and a Math compiler that allows interpreting mathematical terms. also b) the code is pre-compiled and the execution of it is faster after it was compiled the first time. Also they are interchangeable, meaning, due to the meta code in the end is the same I easily can call functions written in language A out of language B.

The Softcode (layer three)

Due to what softcode is, just a plugin it is very flexible, Softcode can be (theoretically) everyting from brainfluck over MUSHCode to mathematical term.

It can be written within the server as parts of attributes or (at one point) as commands and so on.

Last words

So I thought about what would make people use this server, and what not so – and this are only guesses – I made up some pro’s and cons compared to other servers. I could not check any of this really so they are theoretical and what I expect.

Pro:

• adds another layer between hardcode (core) and softcode (layer 3)
• Plugin based so easily expendable

Cons:

• More memory footprint
• Slower core (I’m not sure how plugins are a gain over softcode but I assume so)
• Slower softcode (I don’t know if the metacode speeds thigns up enough to make it a match for pure C code but I doubt it)

Well something new today. Has taken me long enough.

First of all, there is a new server, so rejoice! It’s faster, better equipped with ram and I’ve taken a long way to make it somewhat secure.

And with that we’re already at the topic. Security. Browsing though the log files of the server I’ve noticed a hell lot of stupid script kiddies attempting to brute force their way into the server.
Lines of lines of failed logins to the root account via SSH, hence the thing isn’t even enabled but how shall they know eh? Not to mention 40+ warnings from snort, starting from port scans to attempts to deploy a MS-SQL work on this linux server.

The chance that any of those things get through are low, I know, but eventually they might get a password right for a user on the system. Or I might be to slow to update to the latest patch of software XY and a buffer overflow is open for a day or two.

So simply monitoring with utilities like snort wasn’t enough for me, and while reading the logs I noticed that the attacks always were came in bundles. 3, 4, 20 attempts with different things from one IP before they gave up.

Now I figured, what the heck, that’s enough. Not only that it’s cluttering my logs but heck the 1.000.000st attempt might by chance just hit the right password for the right user, so I’ll put a few more obstacles into the way.

And I ended up with writing my own small ICE (Intrusion Countermeasure Engine), those of you liking shadowrun or cyberpunk will know the term ;).

It isn’t the most complicated part of software just adds the ability to actively take steps against attackers as in dynamically adjusting the firewall rules to prevent attacks from going too far.

It mostly is based on the capability of scanning through logs and reading the interesting information out of them and react to what other applications noticed.

Currently it supports 3 methods of detection:

• SNORT based logs
• Attempts to log in with banned user names
• Attempts to log in with non existing users

All of those are kind of heuristic and might produce false positives as if your users XY attempts to login and 3 times misspells his username – so it is set up to allow VPN connections from any address (meaning the filter happens only for non VPN connections, assuming you don’t let script-kiddies into your VPN.

Well there it goes: RedICE 0.1.0

Currently I’m writing an GUI application, I started developing on my mac using the lovely CocoaRuby framework that is truly a pleasure to use together with the nice GUI builder and the rather good incorporation with ruby. After about a month I got the main part of my app done, the logic behind it, a custom control to add a special feature I needed to the guy loading, saving exporting, I told a costudent of mine about it. He loved the app and told me he wants a copy, first I told him I’m sorry for him but it’s a Mac only app. After a few days it got me that it must be possible to separate GUI and program logic far enough to comparable easily switch from one GUI framework to another (in my case from CocoaRuby to Ruby/TK as from the info I found it was by default included in the windows installer and I wanted to keep things simple for the folks that are sticking with Redmond OS’s). So it took me another week to completely separate GUI and program logic and no single line of ‘program’ was left in the gui parts of the application. All drawing routines were moved into a custom class that encapsulated them in own ones that were supposed to be used in the code so only the drawing class itself providing simple functions hat to be redone not the whole custom class. All logic and event-handing were moved in another class that were created upon GUI start and just the appropriate functions were called in the events of the GUI and some callbacks set up to handle responses. After all that was done I was quite happy, I reached my first goal and it was working DAMN nice, it helped a lot to clean stuff up and refactorate a few things. So two days later I got a TK GUI done that at least handled 75% of the functionality of the original GUI and I went to try it on a Windows machine. I was about to explode when I noticed that the latest ruby installer didn’t had the TK libs provided any more but instead the Fox GUI Framework. While it is perhaps not that a big deal to do yet another GUI for the App there was something that bothered me: There are a load of GUI frameworks for Ruby but NONE is universal for all systems that hold ruby. Some are preinstalled on this, some on the other and some on non systems. Some work here but not there or are a pain to install at all. * GTK+ easy to set up on linux but a pain on windows and Mac * Tk Easy on Mac and Linux but in the newest installer for windows it needs another 20MB packet of ActiveTcl to have the needed libs. * CocoaRuby A dream on the MAc but of cause won’t run on any other system * Fox Is supplied with the Windows installer not with any others I surely forgot quite some toolkits but I doubt it is much different with any others. Not to mention that non of the toolkits I tested come even close to the usual ruby feeling of ‘it just works’. So why did I started with ‘what java has that ruby don’t’ and I must admit it is the first thing like that I find ;) it has a default GUI, one that runs on EVERY system that runs Java, without installing any extra packages, without compiling stuff, without having things working on only one system but not the other. Ruby truly leaks such a thing, like an universal GUI that works wherever ruby works and does so on the ruby way (nicely), without that one has to screw ones mind to write ruby that look nearly like C (or worst!) without all such pain. And not to forget, one that just looks nice (honestly most GUIs especially the ones that look like windows) are just plain ugly. So much for that, if I ever find the time I’m going to try this but I don’t think I know enough about GUIs to get this running - help is welcome!

Hey everyone, I am sure there are hundreds of nice ways to keep track of how many people visited a rails applications. From log analyzers for apache logs, over free web counters to self written code or gems that are floating through the net. And i agree all of them are nice, I currently use the neat tool from google which has a lot of features. So in the first days before I had set it up I was looking for just a simple way to see how many hits the webside had without any nifty graphics, html output or referrer analysis. I was surprised not find anything about it on google right ahead. Well a solution had to be found and as a ruby fan surely it included ruby. The original code I tossed together after a minute or two where (all assuming you’re in the logs dir of your rails app): cat production.log | grep "^Processing" > out.tmp && ruby -e 'data = Hash.new;File.open("out.tmp").each{|l| data[ip = l.split(" ")[3]] = (data[ip] || 0) + 1}; puts data.length' Now it is ugly, using files and all this but it worked which were the main idea behind it (quick and dirty), very dirty I must admit so. A few days later I talked with a friend over the topic and we came up with a lightly nicer way to do thigs: cat production.log | grep "^Processing" | ruby -e 'ips = Hash.new(0); while gets; ips[$_[/\s.*?\s.*?\s(\S+)/,1]] += 1; end; ips.length' While this already was a nice solution (and terrible ugly to read) it still worked and gave the number of hits just as the first line of code. The basic idea in both of this is to create a hash of IP’s that accessed the website and then return the length of the hash. Nice, both of them so in the end there is a much simpler what without the use of ruby (as much as I love the language) so here what came out after a bit more pondering, it don’t uses ruby any more just console commands: cat production.log | grep "^Processing" | awk '{print $4}' | sort -u | wc | awk '{print $1}' A short in detail description of what happens. `cat production.log` gets the data of the content of the logfile itself the `|` pipes the output to the next command (like one would write to the STDIN). The next step is to filter the log `grep “^Processing”` takes care of this it only leaves lines that start with the word Processing which would be (I masked the IP from that line): Processing ArticlesController#permalink (for xxx.xxx.xxx.xxx at 2006-08-01 20:53:29) [GET] The following command `awk ‘{print $4}’` gets the 4th word of the line, which in this case would be the IP of the whole request. Now we have a list of all IP’s that ever accessed the site. As there are still multiple lines from the same IP in there we need to filter them. `sort -u` takes care of this it sorts the list and remove double entries, like twice then same IP. Now all that is left is to count the lines in this file, `wc` is our friend here, it returns 3 values while the second and third don’t really interest is the first does so again `awk` will fetch the word of the resulting line we want. The last command in the chain is `awk ‘{print $1}’`. Done. Enjoy it or not ;) most other tools are more precise, accurate and give more information still it is a quick way to get a rough idea and I found good use of it.

Currently I’m working on a project using Rails and heavy Ajax coding for a browser based application for address, room and personal management. [Ajaxscafold](http://www.ajaxscaffold.com/) is an excellent tool for this and took a lot of ‘routine’ work, by providing the whole edit/new and whatnot, from me. **Step 1** - Getting rails to work. ———————————– Okay to get the easy part done quickl: gem install rails gem install ajax_scaffold_generator rails /path/to/todo cd /path/to/todo rm public/index.html Now we’ve a basic rails app set up. If there are any problems with this steps: [Google](http://google.com) is your friend, there are about one million trillion tutorials on that topic and it might be a good thing to start of with one of this, as I’m not going in details here. **Step 2** - Setting up a database and the Scaffold. —————————————————- We’re going to use migrations for this, actually only one but that will be just fine. Get to the directory we set up the rails application into. And type the following: script/generate migration CreateTodoTable create db/migrate create db/migrate/001_create_todo_table.rb This will create a database migration file we can later easily patch into the tables. Here the content of the migration file: class CreateTodoTable < ActiveRecord::Migration def self.up create_table 'todos' do |t| t.column 'name', :string t.column 'description', :text end end def self.down drop_table 'todos' end end With that done the next step will be to run `rake migrate` to have rake set up our table, and we’re done her too. At this point we’ve an all nice and shiny database, table and a basically running rails, (hopefully) if not just make it work. All left to do is to generate the scaffolds, which again is ‘ruby simple’ to be exact only a single command: script/generate ajaxscaffold Todo **Step 3** - Adjust the templates. ———————————- To add the search features we need to make a few adjustments to the templates, fist we alter the `app/views/todos/component.rhtml`and replace:

<% new_params = params.merge(:controller => ‘/todos’, :action => ‘new’) %> <%= loading_indicator_tag(new_params) %> <%= link_to_remote "Create New", { :url => new_params, :loading => “Element.show(‘#{loading_indicator_id(new_params)}’);” }, { :href => url_for(new_params), :class => “create” } %>

With the new code for search field, we don’t need a form as we want to make it dynamically updating, and use a observer instead of it. We set the update to an empty string so it don’t overwrites any other elements, as we will only return an java script:

<% new_params = params.merge(:controller => ‘/todos’, :action => ‘new’) %> <%= loading_indicator_tag(new_params) %> <%= link_to_remote "Create New", { :url => new_params, :loading => “Element.show(‘#{loading_indicator_id(new_params)}’);” }, { :href => url_for(new_params), :class => “create” } %> <% search_params = params.merge(:controller => ‘/todos’, :action => ‘search’) %> <%= text_field_tag :search %><%= loading_indicator_tag search_params %> <%= observe_field(:search, :frequency => 0.5, :update => ”, :loading => “Element.show(‘#{loading_indicator_id(search_params)}’)”, :complete => “Element.hide(‘#{loading_indicator_id(search_params)}’)”, :url => search_params) %>

Okay so far the first step, but it would be too easy if that would be all. We need to give the footer an ID, so we can later easiely set new paginnation links via javascript. In the same file, at the very bottom change:

<%= render :partial => ‘pagination_links’, :locals => { :paginator => @paginator } %>

by adding of `id=”todo-pagination”` to the `