Git with github

It feels like if you are not using or talking about Git nowadays, then you are out of date. But if you are, then you are definitely cool (do check out this post about from cvs to svn and then to git and the cool factor). So I am cool :-) as I am using git, but the exact reason I started to use git is because of another post: "sharing your code".

Git has a quite different idea from cvs and svn, when you do "commit", the change won't go to centralized server, it stays local. Then when using github, do another "push" the code then will be visible to others.

The idea of local is quite good, you can keep your own change history without begging others to commit your change and meanwhile keep updating with changes from others if you like (in such a case, git can be easily used together with svn). If you want to keep a record of your stuff outside of your local machine, then use github.

That's how I am playing with micolog (as I forked a micolog-cxu) and google app engine now. Before I do update with google app engine, I would do a "git diff" to make sure the files I will update, after "appcfg.py update", I will commit my changes to git and push them if necessary. Later if I want, I could do a simple script wrapper for these steps.

SimpleG theme for micolog

wlsy@g9net published a nice theme called SimpleG for micolog as a New Year's gift, and that's what I am using here right now. Here is the advertisement banner by wlsy:

Not only it's nice looking, elegant and as what author has said - pure css, small and fast, but also it is seamless to integrate widgets like Google Friend Connect, Google Ajax Search Api. Of course, you have to check author's page a little bit. I guess wlsy would publish his complete code if there was no key settings for these widgets.

And it should be easy to convert this theme to wordpress as well, or send a request to wlsy :-)

BTW, I feel for someone, code may be a determining factor to choose what blog/cms/framework system to use (for me too, it has to be written in python); but I have another one, that is the theme, if django could have provided a default one, I would have started to use it two years ago or earlier (with that said, I did develop a tool for internal use based on django admin since the admin has a decent look :-) ). So keep nice theme coming ~

beautiful urls

As I am writing posts frequently these days, I feel troublesome to write both the title and slug manually. So I decided to do this via jQuery:

<script type="text/javascript">
jQuery(document).ready(function() {
    $("#title").blur(function(){
        $("#slug").val( $("#title").val().toLowerCase().replace(/[^\s\w]/g, '').replace(/\s+/g, '-') );
    });
});
</script>

So when the slug is used in url, it conforms to the url structure Google Suggested (lowercase, '-'.join(words)). Of course, there are a lot of assumptions to use it: only ASCII characters are used; there won't be extra space in the beginning or the end of the title (a trim() or strip()* should work, though I have to look up what the corresponding name is in JavaScript); there will be space after punctuation characters and so on.

The assumptions are ok except the 1st one - only ASCII character is allowed. Then today I saw a post about using PHP's iconv function to deal with non-ascii :

function toSlug($string,$space="-") {  
      
    if (function_exists('iconv')) {  
        $string = @iconv('UTF-8', 'ASCII//TRANSLIT', $string);  
    }  
  
    $string = preg_replace("/[^a-zA-Z0-9 -]/", "", $string);  
    $string = strtolower($string);  
    $string = str_replace(" ", $space, $string);  
  
    return $string;  
  
}

This is not bad. I wish I knew or searched such a Java function or library then I can do much less work for my real job. But again this won't work for Chinese characters, maybe a Google translate can help a bit.

strip()*: it's always a fun topic in python, as there may be interesting Google Ads associated with it.

And my github diff for the slug function: http://github.com/xclricky/micolog-cxu/commit/f20bc501068053da9a0ec59a5ecefbb6ae64a93e#diff-1

Single threaded dev appserver

A while ago, I saw some discussion about using google appengine development server as a live (or production) web server, I forgot the conclusion (probably is no), I also found a reason to say no, when I was adding and testing the accept trackback and test trackback function to micolog on local development server.

Basically, when the post is published, it will send out a trackback request to the linked post, and wait for the success or failed response from the trackback handler of the linked post. But in the trackback handler, since I added another check step, to check the original post whether it exists and it has the linked post url or not, the trackback handler will send out a request as well.

Unfortunately, when I was testing locally, I thought probably I can just use the local post 1 to trackback local post 2. Then the above mentioned 2 requests basically were queued in the dev appserver since it is Single Thread. The 2nd request from trackback handler will never be executed until the lifetime of the trackback handler is gone. So locally, the trackback handler could never determine the orginal post is valid or not, the trackback always failed.

And I learned from a hard lesson, the dev appserver is serving requests by queueing them with single thread, and it is not a good idea to use it to serve wild public.

what browser to use, if not firefox?

What if I don't need firebug, and I just need a browser, and I am in linux.

1. Flock is a good option: so you still can have firebug or other web development tools. But it's still Firefox.
   
2. Prism - this is really not a browser, but it's really good as I am using its webapps like Gmail, Google Reader heavily, and I almost have no complaints except one: by default, Gmail doesn't support navigation key (like Alt + "<-" as back). But I can change that through the .prism/yourkey.default/webapps/google.reader@developer.mozilla.org/webapp.ini file, by setting navigation as yes. And probably I can define a lot of webapps in prism, so it could be considered as browser. Maybe I will test it out later. However, you sort of lose the 'Tab' ability, which goes back to the multi-window era.
3. midori - as I am searching webkit related solution, this offers a clickable installation solution in ubuntu. Seems not bad if I just want to browse, though feel like some javascript don't work properly (might be a problem for webkit in general). After I have more experience with it, I can report back.
4. Google Chrome - still out of reach for linux.

Then really, why not firefox, it's frequently crashing for me now. I guess because some of the data (sqlite) file for my firefox is too large now, (e.g., urlclassifier3.sqlite is 51M). And I am afraid if I keep use Flock, it will go to that size as well, then the same problem. Hopefully there will be some nice solution for this, or Google Chrome will be around.

accept trackback

It's not that fair only to trackback outside, but not accepting outside trackbacks. So I modified micolog to accept trackback.

No data model change is necessary as trackback is a type of comment, though a comment type field could be added as what Wordpress does. Only need to add a requestHandler to deal with trackback request.

Based on current micolog setup, I could add function to  the do_action handler (if you are not aware of that yet, try test this: http://cxu.yimudi.org/do/test ). But a separate handler is nicer as I can separate it into a module easily later. So I did that:

http://github.com/xclricky/micolog-cxu/commit/c95b17a3c1bcbf59fff27c1f091cd32c4ace5ea3#L1R395

And this handler will deal with any uri startswith "/trackback/"

 ('/trackback/(.*)', TrackBackHandler),

the stuff after 'trackback' is the real link to the target post. So in the handler, there are lots of checks, to check the post information is complete, to check the target post exists, to make sure the incoming link hasn't pinged before, etc.

But I also added some extra check, used some idea from Pingback (so what I implemented here is not an exact Trackback, which I hope can disappoint spam a bit), that is to check the incoming url exists or not and the content of incoming link has the target post url.

That's it. And let's see whether it works :)

Note, if you want to try to use it, please remember to update the template as well, since you need to add a trackback url for each of your post.

Update: it didn't work as I saved for the first try, because the urlfetch doesn't like non-ascii data, I have to add following to make it work:

result.content.decode('ascii','ignore')

test trackback

How fast Search Engine will include a new site

As I am waiting and checking each day to see when Google will start to index my site, I thought how about test yahoo, live, baidu, and the whatever cuil as well. So I submitted my site url to each of them yesterday.

Today, as I checked, live.com is giving back results by query: site:cxu.yimudi.org. And that's the only one returning results. Hah, pretty sweet and fast.

Just searching query "cxu.yimudi.org", Google, will give back my twitter note and once it gave back the github link; live.com gives back my site and my comment at xuming's post. Interesting! And baidu, yahoo, cuil don't have any results (from now on, I will stop testing cuil, though it has a quite different arrangement of its search result and it claims it has 3 times of info of Google and 10 times of Microsoft).

I will update later to see when yahoo, baidu start to show results.

Dec. 12: Google has the result now, although the result it returns is quite confusing (http://cxu.yimudi.org/tag/search)

use Google Custem Search

I have long heard the Google CSE on demand indexing, but not got a chance to try myself until yesterday. The idea is Google Custom Search can use the sitemap submitted via the Google webmaster tools (Google coop (CSE) will send an email to you first to confirm the request to associate the webmaster account and CSE account), then you can request through the Google CSE indexing tool to index on demand.

This helps a lot for new sites, such as this blog, which hasn't been indexed by Google yet, so if you try query: "google site:cxu.yimudi.org", I guess you won't see anything till ? ( I can update the date later :) -- it is Dec 12, 2008, less than a week, not that bad ). But via the Google CSE, nearly within 3 hours, all my pages have been indexed: try query: "google". Sweet!

And you probably can spot the top search box is powered by Google Custom search now :)

play with micolog -- handle 404

As soon as I set up this blog, I inserted Google ads, and I want Google to index my site asap. I submitted site url here: http://www.google.com/addurl/ , and wanted to submit sitemap as well. Then I was stuck with the verification of the ownership of the blog.

Google Webmaster has two ways to verify site ownership, one by creating a google requested html, and one by adding google specified meta tag. I chose the html method first, however I run into the issue "Your site doesn't return a 4xx HTML status code for non-existing URLs". Then I decided to use meta tag verification. But Google just trapped itself in the previous error, and keeps showing me the same error message even if I have switched verification method.

So, I gave up and decided to fix the problem of 404 page with 200 status. The code of micolog is quite easy to follow and understand. Also the logs from GAE's admin console is very helpful to confirm my guess (although I only checked it after I already made the change and about to deploy). And the problem is after the first visit, error (or not existing) page will be cached as well, and the cache will be served immediately for the 2nd visit even without going through the cycle to set the http status code to 404.

Several solutions come in mind: 1. disable the cache for error page; 2. define a new way to cache error page; 3. change the cache method to cache http status code as well and set it before serving cache.

Actually it turns out method 1 and 2 didn't work well because of the setup of url dispatch and the underlying design of micolog, but it's fun as I read more of the code and find out why it won't work. So I have to change the cache method to include http status. Then I run into trouble to get the http status code, GAE's Response object doesn't offer a way to get the status code. So I decided to fight with it and to retrieve its private field (by using object._(classname)__(variable) ):

status_code = response._Response__status[0]

Another way is to set and store a status code somewhere in the micolog's handler object. But I feel it's better GAE's Response object can provide it. Anyway, here is my current change and it works.

Index: base.py
===================================================================
73c73,76
<             skey=key+ request.path_qs
---
>             if key:
>                 skey=key
>             else:
>                 skey=request.path_qs
80a84
>                  response.set_status(html[2])
89c93,94
<                 memcache.set(skey,(result,response.last_modified),time)
---
>                 status_code = response._Response__status[0]
>                 memcache.set(skey,(result,response.last_modified, status_code),time)