About to End, Again, Again

Based on last year’s blogs, I posted last week’s “final post”. Well, you can’t decide when to end. Based on the weekly feedback notification email, I have to write my weekly posts 13. =_=

Anyway, not a big deal, all I need to do is brag more about the Dashboard with Robby. See, I’ve already written about 100 words and still writting. (I watched The Avengers: Age of Ultron. The line “I’m out of my words”, Thor said while he is holding against Ultron, is really funny.)

It’s been a productive and funny week, and especially the weekend. I delivered my final documentation PR and put a full stop of this year’s GSoC. Though GSoC is over, the project will still go along. I’ll work on Dashboard in my leisure time to make it better, fix those small bugs at least.

About to End, Again

According to the timeline on melange, GSoC of this year is approaching the end. Actually, tommorrow 17th Aug. is the suggested ‘pencil down’ date. The GSoC experience is really great and having two in a row is indeed something that will rooted in my memory.

Like last year I worked on the Dashboard this year, it’s mainly about updating things and fixing bugs. However unlike last year’s rookie works, it’s more about efficiency and whole-structure change. Those modules left last year all have gotten updated.

I’m really not good at the words, so I think this is it. Coding in OpenMRS, “Write code, save lives”!

The First Ping

Few days ago I decided to write a basic PING program to practice my C programing skills and network stuff. And it turns out to be a 2 days work, I really have tasted the difficulities of C programing after I spent hours on location bugs of operator priority and wrongly regarding pointer as struct. Actually by retrospection, apart from these stupid mistakes, this work is not very hard. It’s just that so many new concepts need to be learnt first.

The PING program, as wiki says,is used to measure the round trip time(RTT) of remote and local machines for network diagnosis purpose. This page vividly describes what it does and how it came out.

Basically PING runs on ICMP protocol which belongs to IP protocol family. And it utilizes the ECHO REQUEST, which by definition will trigger the remote to send back a ECHO REPLY packet that carries the exactly same payload sent, packet to measure the RTT. Usually it’s achieved by using the current time as the payload and perform a substraction when we got the reply.

So to implement it, basic understandings of ICMP and Socket are certainly necessary. And most time I spent is to understand them. As I had not gotten a book around, I found a few example sources and studied by researching them. Though I can guess some lines of code literally, it indeed costed me lots of time to study the structs and constants used. If not counting these things, things were fine, I just need to create a right address and find the right protocol then assign them to a raw socket, and then set the proper header like checksum, send it and receive it at last.

C programming requires siginificant patience and carefulness.

GSoC 2015 Week11

GSoC 2015 is about to the end. I’ve been working mainly on things that fix small issues now.

This week I managed to add Travis-CI support, which is commonly used free Continuous Integration services platform, to our project. It will provide you a VM for testing, where you need to provide scripts, .travis.yml, to set up the envrionment. For Dashboard, I need to get the OpenLDAP stuff and Mongo installed. This is where things are getting a little tricky as OpenLDAP installation requires some interactions. Luckily enough, I found out that I can set envrionment variable DEBIANFRONTEND=noninteractive to disable shell interactions, as we can configure it via configuration files later anyway.

My travis works are placed in this repo specifically created for testing.

GSoC 2015 Week10

This is just another week about bug-fixing and html template convertion, same trivial stuff like usual.

I was mainly working on getting OAuth module updated, during which I found out that I’ve made a bunch of big mistakes last year and my code wasn’t deployed… It took a lot of time for me to review the OAuth2 protocol and the packages we were using. I kind of just rewrote what Elliott had written in another superficially different style without thorough understanding of the tools and protocol I was using that time, and hence bugs were left. It was all the fault of laziness and should be prevented in future.

Anyway, the logic has been rewrittened with more careful examination and a functional test, which utilizes request to imitate HTTP requests and cheerio to manipulate HTML document in server side, has been added.

A Severe Potential Bug

While I was doing the testing, I found out a potential danger of Mongoose searching. Usaully we do searching like Model.find(conditions) where conditions is a Object containing (key: value) pairs that we want our documents to match. However, if some value is undefined, that pair will be ignored.

A very bad example:

I want to search {foo: bar}, however, as bar is undefined it turns into Model.find({}), which will return every existing document of that collection.

What a terrible thing! You can’t be careful enough to do the DB querys, and I’m filtering the Dashboard now. There is better none of this potential vulnerabilities.

GSoC 2015 Week9

The restructuring work continues, this weeks I’ve mainly done some ending works of this, including:

  • Refactoring modules related with email-verification. Email templates now share a common layout, actionId and verifyId are replaced by a single uuid, few variable names are updated and some packages as well.

  • Reorganizing admin page related things, rearrage the middlewares so other user-modules won’t bother privilege problems which makes things neat.

  • Updated modules to coform to new Dashboard.
  • More…

While I was doing the EJS 2 JADE works, I once again encountered a problem. Elliott had written this code below,

<script type="text/javascript">
$(function() {
    var paramRow = function(){
        return '<tr><% for (var j = 0; j < columns.length; j++) { %><td>'
            +'<input type="text" class="<%=columns[j].type%>" placeholder="<%=columns[j].value%>" name="'+($('#params tbody tr').length-1)+'-<%=columns[j].name%>"></td><% } %></tr>';
/// more

Basically it utilizes some code to generate the Javascript code that will be used in our pages, this section of Javascript would be a plain text block in JADE syntax. However, I can’t figure out how to iterate within a plain text block, which seems not to be a good idea.

Having tried lots of ways and considered the concept of static compiling, my solution are,

  //- if only I could use jade.render in templates
  var rowAdded = '<tr>';
  for (var j = 0; j < settings.columns.length; ++j) {
    var col = settings.columns[j];
    rowAdded += '<td>';
    rowAdded += '<input type="text" class="'+ col.type + '" ' +
      'placeholder="' + col.value + '" ' +
      'name="{{index}}-' + col.name + '">';
    rowAdded += '</td>';
  rowAdded += '</tr>';
//- script start
  $(function() {
    var paramRow = function(){
      var index = $('#params tr').length-1;
      //- unescaped
      return  '!{rowAdded}'.replace(/{{index}}/g, index);
  /// more

This is quiet ugly… but it’s the best solution I could get.

GSoC 2015 Week8

It’s a taxing week indeed, I’ve done lots of works, including:


The EJS template engine is somehow outdated and lots of functions we’re utilizing in Dashboard currently is abandoned, like layouts and partials. Besides Dashboard hasn’t done well in segregation between views level and controller level, lots of html stuff are injected programatically, which makes things hard to maintain. So we’ve decided to convert EJS to JADE, and update other related stuff as well.

Things like this are usually boring and taxing, however, we must accept the fact and conquer it.

Restructing the Project

We used a system modules structure last year, trying to separate the routes logic to make things clearer. But rethinking it again, now I find the idea is very stupid, it hasn’t make things more clearer, but make the code more cumbersome. So I’ve taken this chance to remedy the errors I commited. The structure now mainly borrows Keysone’s, though we haven’t adopted its DB solution.

GSoC 2015 Week7

Although the plans were usually fancy, reality is not sometimes. We were planning to embrace the Keystone.js in this year’s GSoC. It is me that proposed this as I’ve found out that Keystone.js is very easy for a quick development of Node.js centered project and made a small experiment in thisrepo, and later found out that Keystone.js isn’t that fantastic as it first looks like. Like many open source project, it’s still in a very early stage and not suited for certain scenerios.

In my experiments, I doubted once that the Keysone.js maybe have done too much on the Data level, where Keystone.js tried to wrap Mongoose to extend its functionlity, like providing pre-defined tags types and other things up on the top level of development, but hasn’t provided good documents. That time I thought that it’s not a big deal, just some small faults. But later in the development of Dashboard, through few days massing up I find that it’s even hard to define a Array of String, which hasn’t presented until very recent.

The wrap it has done for Mongoose, in my opion, instead of extending it, but limited its flexibility. I really think that it has violated a principle, tools should be used as it should be. Mongoose is designed for flexible and stable data modeling, and focusing on it solely. But for now, if I want to use a custom data type in my model, I have to hope that Keystone has already provided it or patch this kind of type myown. It’s obviously not a good idea, IMHO, a good extension should work along with original tools not covering it.

Anyways, we’ve abandoned Keystone’s DB solution officially. However, its folder structure is still worthy of referring.

Above was all the big talk of the project. For works in detail: I’ve been working on updating the project entirely, and now on EJS to Jade with other view level transformation.

GSoC 2015 Week6

… From the GSoC timeline showed on melange, I thought this week is used for resting… So shamefully, I haven’t done many things this week…

The project is now appraching a important stage to improve embrace the Keystone. Though issue on JIRA said that it was used to simply replace Formage, there are lots of works besides this. It’s not simply a new admin panel, but a new project structure. The current one haven’t tackled the segragation well, many modules are strongly cohesive and the logic flow between sources is like a mess. Last year I’ve been focusing on minor-scale stuff, and this year on big pictures, just to make the Dashboard more maintainable and more easy to comprehend for future developers.

Currently I was trying to add keystone the panel to Dashboard, however small problems were encountered. Here is a issue I opened.

GSoC 2015 Week5

It’s about the midterm evaluation. Things have been proceeding fine thus far. Like last year I’ve made a report and published here. It’s been kind of taxing to record it as my English isn’t very good and I haven’t fully recovered from pharyngitis. And sadly few days later, it got severer again… :/

Besides the report, I’ve also mainly delivered:

  • bug fixes for ID-89
  • more work about ID-88