The Coding Journal of a Cyberfox

A son.

2008-01-02T10:34:46Z

Greetings,

I have an important personal announcement!

My wife and I have a baby on the way; he is due in early May!

(The linked post also includes details on what I'm working on for JBidwatcher2.)

As I say in the post, we're incredibly excited, and a bit scared. It's going to be an amazing journey.

I'll post more about it later, but it's...well, it's been incredible and wonderful, and terrifying and astonishing.

I'm not entirely sure I'll know how to be a good father, having lacked that particular role model in my own life, but I'm sure going to try.

The joy and thankfulness of a new year is upon me, and the vast majority of it surrounds my wonderful wife and our growing baby-to-be.

Happy New Year to all, and best wishes for a great 2008!

-- Morgan Schweers, CyberFOX!

View this post on my main blog.

TDD: The 'Logans Run' of Software Development...

2007-10-06T03:09:24Z

Greetings,

I want to start by making it clear that I know why testing is good, and that it's really important, but I think that the TDD proponents are glossing over the most difficult part of a project.

I would very much like someone to address the issue of modifying code that is not new, and not already perfectly tested (or even completely specified!). That is to say, the vast majority of actual code out there.

TDD is intensely focused on the early development phase (or at least TDD proponents are), and on writing new code, as opposed to what the majority of software developers actually do; maintain and update existing code.

It's really straightforward (and fun!) to write entirely new code in the TDD fashion. I've done it for about 3 decent sized projects now (one Java and two Rails), and it can be really pleasant, and a great focusing tool. No arguments there; when you do it from the start, it's really wonderful.

On the other hand, when you're making incremental changes here and there throughout a very large, pre-existing, only partially tested codebase, it's vastly less pleasant to try and do it test-first.

The Ruby Autotest tool is not such a pleasant tool at that point. You stop wanting to write failing tests, because fixing it means autotest is going to try to do a full retest, which sucks for developer flow... Even if your tests take 'only' 5 minutes to run, breaking a test makes you wince, and writing a failing test and then fixing it is for masochists only (and ones who want to miss the project milestones at that).

The focus of every presentation (this one included) I've seen on TDD being on the start of a project makes me wonder why nobody's talking about later in projects...

Nearly every developer out there is going to face a large codebase with poor testing coverage, and will have to make changes that aren't entirely new code, to existing code that isn't entirely tested. Does TDD have a solution for the 'large, crufty codebase', or is it suited only for 1.0 versions, small projects, and projects that were TDD from the start?

This isn't really a rhetorical question for me. I really want to get my organization's culture more oriented towards testing. I've got buy-in from lots of people that when they're writing new modules and services, they'll do it test-first (or at least 'test around the same time as the code', which is all I can ask for at this point), and that's great. But has anybody developed any tools to make TDD better suited to maintenance and improving existing code?

-- Morgan

p.s. I'm skipping BDD entirely, because BDD is so hardcore in the 'only for already very well specified solutions' camp, that it's meaningless for this question. I'm also using 'TDD' and 'test-first' interchangeably, and I probably shouldn't be.

p.p.s. The title refers to how in Logans Run, everybody was destroyed at 30, so there weren't any old people. In the world of TDD (or at least TDD presentations), there are no old projects, every one is fresh and new, so the issues that come with an old code base are never addressed.

View this post on my blog

My response to eBay's Bid Assistant...

2007-05-19T03:37:31Z

Greetings,

eBay recently launched their Bid Assistant, which acts similarly to JBidwatcher's Multisniping feature, except without the sniping.

None the less, it's good to see them adding features JBidwatcher had six years ago. ;)

Being serious for a moment, I'm actually really happy to see them do this. It's a straightforward feature, and one they should have had long since. If they make it easy to use it'll increase bid volumes, and thereby their end revenue. Speaking as an ex-employee, and a shareholder, this is great!

I always considered multisniping a 'good for eBay' feature, because it meant one bid could get applied to a number of items, without the user really intervening after the initial setup.

It's also a hint that they might still be putting new ideas into their platform, which I'm very happy to see. It also ups the ante for programs like mine, to add more extensive algorithmic bidding. (e.g., 'If I win this, THEN put a snipe on that, because if I can combine shipping I'd want them both...')

What it is not, is sniping. Specifically:

You cannot schedule bids to be placed at a specific time.

The first version of JBidwatcher that included Multisniping was December 16, 2001, and is the earliest implementation of bid groups / multisniping / bid assistant functionality that I know of. It's not world changing, but I'm proud of it. :)

Best of luck with your auctions!

-- Morgan Schweers, CyberFOX!

View this post on my main blog.

JBidwatcher 1.0.1 is released

2007-04-30T01:26:03Z

Greetings,

I've put up the latest version, 1.0.1 of JBidwatcher. It's mainly a bug fix release. It includes a few new features towards better documentation, error messages, and recognition of eBay states, and a new (still completely optional) approach to the eBay affiliate idea.

One of the important fixes has to do with a wording change; eBay changed 'You have been outbid' to 'You've been outbid', and JBidwatcher didn't handle the wording change, reporting it as an 'unknown error'. Also eBay changed the text on their captcha pages, so JBidwatcher was failing logins for unknown reasons. Now it should be better at recognizing those situations.

There was a sporadic startup failure when sorting by a currency-based column with unknown values in it, and saving and restarting. I hope to have addressed that as well.

Very little UI has changed, although the internal FAQ has been enhanced a little, and a whole new FAQ has been put on the web site. There is also a new community forum area that's much better than the old SourceForge forums. In fact, SourceForge is deprecated for everything except bug reports, feature requests, and CVS access to the source.

More information is available at the JBidwatcher 1.0.1 changelog.

Windows users can download a Windows executable! (No installation required, it just runs!)

Mac OS X users can download an app.tar.gz.
Download (it'll automatically decompress) and double-click the .tar file to extract the Application!

You can also download the Java binary for any other platform, including Linux and Solaris.
Launch with: java -Xmx512m -jar JBidWatcher-1.0.1.jar

As always, you can download the source and build it for yourself.

Best of luck with your auctions!

-- Morgan Schweers, CyberFOX!

View this post on my main blog

JBidwatcher and CyberFOX status update

2007-02-14T01:29:55Z

Greetings,

A concerned user recently asked me how I was doing in the aftermath of the issue with eBay sales of JBidwatcher, specifically:

You seemed pretty depressed about it in your post to the website.

I was.

There was a really bad week there, while I was dealing with all of it, back and forth, and just feeling like crap. I got a lot of user feedback, from a LOT of people, that reminded me, as the concerned user put it, not to let the few jerks make me give up.

I took some time to work on other projects, and I've been fiddling with the next major rev of JBidwatcher, mostly cleaning up the code, improving the source layout, fixing small things that nobody else will likely ever see, and writing silly features just for the fun of it. (Like making the internal webserver take 'events' to be posted to the various subsystems, so you can add an item, do a bid, or even tell it to fire off a sound effect through a REST-ish interface. You could theoretically 'script' JBidwatcher through that.)

I've also mostly moved the code base of JBidwatcher to Java 1.5 (mmmm, tasty generics!), since 1.6 is now out. I've also been experimenting with including 'Derby', an embeddable (in the 'ship with program' sense) tiny SQL-based database, so that JBidwatcher's memory usage doesn't grow at the same rate as the number of auctions. Also so that it can offload completed auctions, so they're not kept in memory anymore at all. Yet another thing I've been playing with is including a scripting language (something simple) which would get run on certain events, which would allow for making some of the complex rules people have requested as features. I've also written up an FAQ I need to publish on the site. (The first question addresses my inability to answer emails consistently, in fact!)

Anyway, all told I (and work!) have been keeping myself busy, albeit quiet. It all helps me get past the issue with the people selling JBidwatcher. Future versions will probably not be open source, however. :( I may expose the source, or open certain sections, but almost all open source licenses explicitly allow what those folks were doing, and I've determined that it's beyond what I'm comfortable with. One of the things people repeatedly said in private emails was that the open source nature of JBidwatcher was not critical to their appreciation of JBidwatcher. This means I'll need to extricate myself from Sourceforge in various ways, and cover my own purchase of IntelliJ IDEA, but I think donations will have covered that.

I don't want to charge for JBidwatcher; I prefer people using it and deciding for themselves what it's worth to them. Plus, because it's scraping eBay, I feel bad about asking for money for something that could break the next day. So I expect the program will continue to be no cost. I'm thrilled to get donations, of course, but I don't build JBidwatcher to make money; I have a day job for that. :)

At the same time I'm working on building other projects so my morale won't get torpedoed so badly when someone messes with the sole project I've been working on.

One of the other projects I'm working on are a health tracking tool (weight, blood pressure, hours slept, water drank, steps taken, foods eaten with nutrition information, and more stuff like that, with pretty graphs and sparklines (my weight trend: )). The other major one is a comprehensive multi-user outliner tool. Both are entirely web based applications, unlike JBidwatcher, and both are in Ruby on Rails.

I'm sorry that I haven't been dedicating more time to JBidwatcher, but it's been fundamentally working okay recently, and I needed to blow off steam by doing cool new stuff. So I've been letting it percolate, and rekindling my coding passion by working on other interesting problems.

That's the status as of now; I hope that this sheds some light on my thought processes, and what I see in the future for JBidwatcher. One important thing to take away is that YES, there is a future for JBidwatcher. :)

Thank you, every one, who wrote me, donated, or just thought well of me during all this. I appreciate it a great deal more than I can express.

-- Morgan Schweers, CyberFOX!

View this post on my main blog.

Firefox2 + GMail == Memory Leak

2007-01-15T22:00:45Z

Greetings,

Just noting that I closed my 'gmail' window, and my Firefox process went from 883M to 585M.
I guarantee that 300M weren't being used by the gmail page as it stood. Perhaps Google's Javascript foo is not as powerful as they'd like, and it's holding onto old AJAX results?

I can guess a few other things about what might be going wrong (caching AJAX requests in the browser, for instance), but one way or another, if you have memory problems with Firefox 2 and you keep a gmail tab open, try closing your gmail tab now and then.

I have to wonder if it's a general AJAXy problem with Firefox, or specific to gmail...

One way to test (which I'm going to try soon) is to try a 'reload page' instead of a 'close tab', to see if it's the browser/page-renderer/caching keeping the memory occupied, or the JavaScript on the page that is abusing memory.

There are limits to how much CPU we allow any given page of JavaScript to have; I'm surprised there aren't any limits to how much memory it can have.

-- Morgan Schweers, CyberFOX!

View this post on my blog

Pick your need, pick your tool.

2007-01-10T02:25:10Z

Greetings,

I had a friend recently who asked about what programming languages they should learn. He primarily works as a system and network administrator, and had been bombarded by 'Learn Ruby!' from a bunch of evangelists recently. I assured him it wasn't necessary, and came up with this interesting list.

What language you work in really depends on what you need to do. In my definitely not so humble opinion, here are the best of breed languages/platforms for each of a set of needs, if you just want to Get Stuff Done.

Applications - Java. It's cross platform with rich libraries, and 1.6 makes it even better.

Glue - Perl and bash are best for gluing things together. On Windows this category doesn't really exist.

Web - Ruby on Rails is the best tool out there right now. You also need Javascript, CSS and HTML.

Speed - C++ is best when it must be fast, but usually in service to another language, except for...

Video Games - C++, because speed often trumps other considerations.

Database - If you touch a database, you need to know SQL.

Applet - Flash/ActionScript is the best applet language right now.

Embedded - I believe C is still the gold standard, but I haven't done embedded development in a while.

Mobile - Java ME appears to be the only platform-agnostic answer currently, but again, not my field.

The above is based on my knowledge, as of right now. I can make a strong argument for each of them, but that would require being negative in various ways about other languages for comparison. Instead I prefer to note that other languages or platforms may still be good for each need, but these are the best.

Just like Ruby on Rails exploded onto the scene, this list may not hold true in a week, so it's more a 'snapshot' than a 'bible'. It's also based on my knowledge of what makes for good software development, balancing (depending on the need) features that make development less error-prone, speed, cross platform capabilities, maturity, match between the development (and mental) model and the need, available tools, etc...

Hope it's an interesting view into my development mindset.

-- Morgan Schweers, CyberFOX!

View this post on my blog

It's done. JBidwatcher 1.0 is released upon the world.

2006-09-05T08:16:11Z

Greetings,

JBidwatcher 1.0 is released after 6 years, 5 months of development.

-- Morgan Schweers, CyberFOX!

Like a car knows when you have money...

2006-08-25T04:00:14Z

Greetings,
Software knows when you have time.

I recently made the comment to a user and developer that it was particularly good fortune that I'd gone this long without an eBay breakage. It let me spend some time improving my other technical skills, and coming up with a plan to take JBidwatcher forward, as well as recently lent me the time to figure out some of the longer-standing bugs.

Sadly, that comment must have been 'heard' by the PTB. (I'm really just kidding, I'm not that superstitious. Any timing would have been bad, but it's always something...)

Unfortunately, yes, JBidwatcher is currently non-functioning for the majority of eBay auctions. It will likely still place snipes, but if you can't add new items, that doesn't help much.

I'm working on a fix. Happily, I *just* got my new desktop (my old one's motherboard died, so I punted and went with a completely new build, instead of repairing 'one more time' my old frankenbox). I've got the source up and building, and I've seen the bugs relating to the XML exception and Shipping failures. I've got one fixed (the XML exception) and have gotten a patch from a friendly and helpful user for the shipping issue. I'm checking to see if it fixes it reasonably right now.

As usual, this fix will also incorporate a few other changes I've made since the last release. Hopefully they should all be for the better, though.

Screen location and height should be preserved better now.
Tab removals should stick better now.

I'm looking into a set of bugs reported by another user that stops QUIT from working. (Not the out of memory issues; that's something else, and requires an architectural change, which is post-1.0.) Similar bugs also afflict startup, sometimes making it fail.

All told I'm hoping to do a release this weekend to fix all of this at once.

-- Morgan Schweers, CyberFOX!

It's 'official', 1.0pre6.

2006-03-31T12:20:27Z

Greetings,
1.0pre6 has been released, with the few minor issues that were reported mostly fixed.

Not everything is perfect, there's still some cosmetic issues, but we've been without an official release for TOO long.

All the download information is on the homepage at:
http://www.jbidwatcher.com

Enjoy!

-- Morgan Schweers, CyberFOX!

Letting 1.0pre5 peek out the door...

2006-03-29T14:01:36Z

Greetings,
I've put up 1.0pre5, a new interim release, that fixes the majority of issues that I've been seeing (get selling, get my ebay, multisniping, etc.), and has some other changes that are more subtle.

This is, as always, a pre-release. Please be careful about using it, and please let me know how it behaves. If nothing is glaringly wrong, I'll be releasing it 'officially' in a few days.

For Linux and other platforms:
http://www.jbidwatcher.com/download/JBidWatcher-1.0pre5.jar
remember to run it with:
java -Xmx512m -Xms256m -jar JBidWatcher-1.0pre5.jar

For Windows, download from:
http://www.jbidwatcher.com/download/JBidWatcher-1.0pre5.exe
It's self-contained, doesn't require installation, just run it!

For Mac OS X (10.2 and later), including Intel-based:
http://www.jbidwatcher.com/download/JBidWatcher-1.0pre5.app.tar.gz
It may download as a '.tar' file, and you'll have to double-click that to extract the actual app. Be sure to toss your old copy first, though, otherwise the new one will get a name like 'JBidWatcher.1'.

Please everyone, the most important thing is to get more feedback, so that I can know whether it's safe to release it to the general public (as opposed to the small percentage who read my blog).

Thanks very much, and best of luck with your auctions!

-- Morgan Schweers, CyberFOX!

Quickie beta version to fix current eBay breakage.

2006-03-09T13:49:08Z

Greetings,
Normally, if I could, I'd address this on a weekend, but that hasn't been an option this time, as the breakage was severe enough to require an emergency release.

I'm asking you to test this version, so that I know it works... I've pushed up a new version of JBidwatcher for download, which SHOULD fix the adding/updating issues. Please check this version out first, so that I can feel comfortable pushing it to a larger community. Responses will be best noticed if they include 'BETA' in all caps in the subject line someplace.

The versions are as follows...

For the Macintosh, download from:
http://www.jbidwatcher.com/download/JBidWatcher-1.0pre2.app.tar.gz

For Windows, download:
http://www.jbidwatcher.com/download/JBidWatcher-1.0pre2.exe

For Linux, or any other platform that can run Java .jar files, download:
http://www.jbidwatcher.com/download/JBidWatcher-1.0pre2.jar

I hope these versions work cleanly and perfectly out of the box for you. If not, as I said above, please report to me anything at ALL that you find not working quite right.

Best of luck with your auctions!

-- Morgan Schweers, CyberFOX!

eBay Italy issues and Mac OS X 10.3.9 launch issues.

2006-01-18T11:31:13Z

Greetings,
As I've just put on the home page, there are two currently outstanding issues.

eBay Italy has put up an interestitial 'continue' page on many auctions, which breaks JBidwatcher's ability to snipe and bid. There is a beta for this available based on your operating system. THESE ARE NOT OFFICIAL RELEASES. Do not propagate these, or announce them, or anything like that. They are solutions for users who are having issues with eBay Italy.

Mac OS X 10.3.9 and QuickTime 7.0.4 combine to break JBidwatcher's launch process. You have a number of options:

Downgrade to QuickTime 7.0.1
Upgrade to Mac OS X 10.4.4
Download the Mac OS X JBidwatcher above.

Please note that the above link is for a very, very beta version of the next release. Lots of things are changing, and some things may not even work. It is for those used to bleeding-edge software only.

Best of luck with your auctions!

-- Morgan Schweers, CyberFOX!

Calling strong-stomached beta testers...

2005-08-18T06:48:25Z

Greetings,
I'm testing a new sniping core. The goal is to extract a LOT of the behavior which takes time to a period prior to the actual auction completion, including logging in, getting the item page via the affiliate link, and submitting the initial bid getting to the confirmation screen. It leaves off there, and then exactly when the snipe is supposed to fire, it sends the confirmation form to eBay.

Yes, folks have suggested something like this for a while, but I had to build a framework around it that supports the concept of sandboxing every snipe into its own little world.

Some example problems that could be run into due to the new core include:

Multiple snipes too close together could cause eBay to behave strangely, because each one is an entirely separate logged-in session. If eBay has trouble with the same user logged in multiple times from the same machine this might not work. Untested!
I don't know how long the confirmation page's form is 'good for', so too much lead time will expire it, and too little ends up in the same state we're already in. I believe it should be good for 5 minutes at least, and I'm shooting for a 2 minute lead time, so it should be okay. Untested!
My code may just fubar. ;) It's a seriously major change for the sniping process, and I may not have moved all the logging, and other details, over. It's brand new code, too, so it might just not work right for everyone.
The 'updating' context doesn't share a cookie with the sniping context, so if the sniping invalidates the updating cookie because of doing another login, then it's possible the updating thread will fail to load user-specific data, and/or stop working with items in the Mature Audiences category. This would be a side-effect of eBay's code not being happy with multiple independant logins from a single machine. (It would be as if you were logged into eBay from Firefox, Internet Explorer, and Opera all at once, and doing different things in each browser. I don't actually know if they support this.)

Snipes may just fail with this version.

If you're okay with that, and really want to test, please drop me a line, or reply here with contact info.

-- Morgan Schweers, CyberFOX!

And the crowd goes wild...

2005-08-09T00:20:05Z

Greetings,
*sigh* So eBay, taking note of the imminent release of JBidwatcher, changed their adult login process. I kid, I kid... They did, however, make a rather annoying change to the final page of the adult login process, which broke JBidwatcher uncomfortably.

So adult logins are broken again... It's one of the more touchy parts of the program, unfortunately.

THIS time, though, I don't have to repeat myself to everybody and their brother, letting them know it's changed, and what to do. I've pushed a new configuration (recognizable by 0.9.7.1 as just an update, recognizable by 0.9.8 as an update + configuration changes), which sets 'ebay.adult' to false, and puts up a message explaining what happened.

As soon as the configuration is checked and loaded (some 10 seconds after launch) everybody will be updated with the newest changes. It doesn't FIX the problem, because the change is a bit more convoluted than can be handled via the automatic string update, but it does make it so that anybody currently running into it will not be running into it anymore, unless they are trying to bid on mature audiences items.

If they ARE trying to bid, snipe, or monitor mature audiences items, then they'll need an interim version, while I work on putting together 0.9.9.

-- Morgan Schweers, CyberFOX!

JBidwatcher 0.9.8 is released (Fly! Be Free!)

2005-08-08T09:15:18Z

Greetings,
...and it's away!

JBidwatcher 0.9.8 is released, with fixes for the Search and Drag-and-Drop problem (eBay's URL change), and more...

Straight out of the changelog, here's the latest...

Fixed eBay's URL format change that broke search & drag-and-drop
HTTPS Proxy support in the Configuration Manager
System Tray support on Windows
New customizable columns: Location, Feedback, Feedback %
Speedups for bidding and sniping
PayPal Icon for auctions which appear to support PayPal payments
Some fixes for eBay changes can be done w/o having to download a new version.
Time is synchronized on a more consistent schedule
Sign-in failures are recognized during bid attempts
Much more detail on each step of bid/snipe attempts
You can place bids/snipes over the BIN price if you really want to
You can disable .html.gz archival of the auctions you've monitored
Several other minor bug fixes and code improvements

From a coding perspective, I'm starting to seriously use the TQM (TimeQueueManager) code I put together several versions ago, which uses a heap to manage a queue of items that will occur in the future, and instead of doing them itself, it requeues the events onto their appropriate queue. It's so cool that I named it 'SuperQueue' in the thread names. (One of the code changes this version is that I actually named all the threads, so debugging and such, tools will see names that make sense instead of 'Thread 5'.)

I eliminated the hackish way time was being synchronized; every time the 'time data' was being checked, it was checking if it needed to update, which put logic all OVER the place, and made it non-deterministic when the time would update. Instead the time is now synchronized every half hour, firmly. This stripped out a bunch of code, which always makes me feel happy.

The System Tray feature, a benny for Windows users who have been feeling neglected as I focused on making JBidwatcher work especially well on the Mac, works in a particularly hackish way. It actually extracts, via homemade code, a .DLL from the .JAR that it ships in, and puts it into ${user.home}/.jbidwatcher/platform/tray.dll and then loads it as an external library.

So JBidwatcher kind of acts as its own installer, now. I think one of the next things I'm going to have to do is provide an 'Uninstall' menu item, that gets rid of all the cruft in ~/.jbidwatcher.

With this version, and the way things appear to be cleaning up a whole lot. I need to rebuild the screenshots on the homepage, including multi-platform shots, and document everything that's shown, and turn that into a PDF or something that can act as a manual, and then maybe I can think about calling it a 1.0.

For all those who want to know what's coming, here's the list of things that I put off as potentially being '0.9.9':

Allow direct machine-to-machine synchronization
When network fails after waking (sleep wakeup), pause the network-related threads for 10-15 seconds, so it doesn't spam the logs with network failures and mark items as unreadable
A 'Properties' sheet for tabs, which allows setting the tab completed items are moved to, the background color, and anything else I can think of.
Document how to use a USB pen-drive with JBidwatcher. (One of my ingenious users figured this out, and I think it should be shared...)
Synchronize auctions.xml to a 'known location' on the filesystem, so tools which can't find it in .jbidwatcher can do their own network synchronization, and potentially allow loading from that location too.
Show how many bids you've made through JBidwatcher, either under the affiliate program or not. (Some users were curious, and I think it's a cool stat.)
Save search window location and size, and maybe the same for the config window and which tab is open.

Several people have recommended recently using JNLP. I plan to do that, using the extremely nice collection of JNLP fixes supplied by Nick Minutello, but it will have to wait until I go to my own server for web hosting, instead of being on Sourceforge. I've paid for the new hosting already, and am configuring it as I find time. The switchover to the new 'jbidwatcher.com' server should be completely transparent, and will allow me to do a few tricks I've always wanted to do, like allow 'bugs.jbidwatcher.com' to go straight to the bug list. I will still be using Sourceforge's bug and feature tracking, and forums, etc., of course, I just feel the need for more control over my own web server (and it's necessary for JNLP hosting, imo).

I'm also going to admit something, since I've been testing timing heavily lately. The affiliate program loading code does slow down bids a bit. Specifically, in recent tests, doing a snipe/bid with the affiliate program turned on took 3-4 seconds to pre-load the item, another 1-2 seconds to do the bidding, and then 4-5 seconds to load the post-bid data. Now the post-bid data is important, but by that point the snipe is already done. So the actual snipe takes 5 seconds, but it feels like 10. Now, with the affiliate program off, it takes roughly 1-2 seconds to place the bid and check the status, and it's done. So sniping and bidding via affiliate program feels a lot slower, but in truth it's only a few seconds slower for the critical part. (If you're shooting for the mythical 1-second snipe, I hate to admit this, but I've never gotten a 1-second snipe out of JBidwatcher, although I haven't tried in around a year.) Anyhow, JBidwatcher tries to adjust for the extra load time, but don't try for 1-3 second snipes. Leave at least 5-7 seconds of buffer, and I recommend 10 seconds for those on the fastest connections. I use 10 myself, usually, and have never had it fail to bid in time, and really rarely with enough time for anyone to respond. I'll see if there isn't some way I can asynchronously post-back the result of the snipe, while it's loading the post-bid page in the background.

All this is with the new speed changes I made in this version. The primary fix was to make it so JBidwatcher doesn't try to update any other items during a bid, so there's no network conflict with itself.

Anyhow, that's the news, and I'm outta here... Enjoy the new version, the new features, and as usual let me know ASAP if there's anything wrong!

-- Morgan Schweers, CyberFOX!

JBidwatcher 0.9.8

2005-08-01T10:01:03Z

Greetings,
Just a note; I believe you're in for a treat with 0.9.8. It's got a LOT of cool new features, this time even some for the Windows users out there. Hmmm, could it be the single most requested platform-specific feature ever...? I'm really ratcheting in on 1.0, and if I can crank out some good documentation, I might be there before the end of the year. :)

Some noticable new features in 0.9.8 include:

The ability for me to put up 'fixes' that JBidwatcher automatically consumes, for when eBay changes something that can be fixed with just a config or string change.
I can also put up one-time messages to alert of problems w/o putting out a new version.
Updates are put on hold while sniping or bidding is being performed, which should speed up sniping/bidding some.
There's a 'PayPal' icon for auctions which appear to support payment or deposits via PayPal.
We have some new custom columns: 'Location', 'Feedback %' and 'Feedback' (of the item/seller, of course).

...and obviously, the fix for searching and drag-and-drop.

Unfortunately, due to some issues with Sourceforge, I didn't get the version out tonight. It'll take me a bit, because my time management sucks during the week, so it might be Monday or Tuesday night before I get to release it. It's not an emergency, since bidding and sniping still work.

-- Morgan Schweers, CyberFOX!

eBay's changed the URL format their search and browse pages generate.

2005-07-28T23:20:28Z

Greetings,
*sigh*

eBay changed the actual URL format. It's encoded a little bit, changing things like '&' to QQ and = to Z, for instance. The number is still in the URL, but for some ungodly reason so is the title.

This seems a really silly thing to do, but they've gone ahead and done it, so I've got to work up a new version to fix it.

In the mean time, adding items by item # works. SO FAR it appears that adding items to my Watch List, and then synching with eBay, then clearing my watch list, also works.

Seller Search also appears to be unharmed yet, but it's only a matter of time for both of those workarounds. Normal searches and URL searches are both broken, and drag-and-drop from eBay's URLs is likely to also not work.

Bidding appears to still be working fine, so sniping should also be working.

It's the summer months; things are slow at eBay, so they take advantage of that and make a lot of changes. Some years their changes don't affect JBidwatcher that much. This year, it looks like it'll be a little less easy-going.

I'll try to get this resolved this weekend. Until then, add auctions by item #, mainly.

-- Morgan Schweers, CyberFOX!

JBidwatcher 0.9.7.1 - Fixing the latest eBay changes...

2005-07-13T09:20:23Z

Greetings,
Amidst the deep thoughts comes a cry; 'Release the Code!'

eBay changed their title format for the first time in a VERY long time, and because some of the data is extracted from their title, that broke JBidwatcher's ability to load and update auctions.

Visit:
JBidwatcher's home page for more information.

It's a mandatory upgrade; by the end of the week, 100% of eBay's servers will be running the new code.

-- Morgan Schweers, CyberFOX!

Journalism, bloggers, and how to fix comment and blog spam.

2005-07-10T09:35:54Z

Greetings,
Well, let's see how well Robert Scoble's comment-replacing search engine works. He wrote a few things about how bloggers will have to become journalists, which I basically agree with. He also begged for someone to save him from blog-spam.

As regards 'journalists', I wish I knew where it came from, but it was once pointed out to me that 'The news always appears true and complete, unless it is about something you have personal knowledge of.' It's never been driven home so forcefully as when I worked in the anti-virus business during the time it was a tech darling of the news media. You start wondering just how much of the rest of the news is as incomplete, and misinterpreted as the part you KNOW they're getting wrong.

With bloggers, because it's more first-person reporting, you have a better chance of being able to, as Scoble said, triangulate onto the truth. However, with blogs, you have the 'echo chamber' to worry about, where the people you pick to read have too many traits in common to get a good triangulation. This may mean you have to read some things which really piss you off in order to get a complete triangulation.

On the other issue, I believe I know how to solve the blog-search-finding-spam issue, but the solution is too long to fit in these comments. ;)

It's also not perfect, but merely 'good enough', I believe. Sadly, without data I can't test it. I don't have the resources of a Technorati, or such, so as per the joke above, my fix may turn out wrong, but still point the direction for a more complete fix.

I believe the core of the solution is statistical natural language processing, 'voice' analysis (as in turning into numbers the feel of the voice being used by the writer), and consistency mapping and comparison. With the right balance, it could at least provide another variable to sort on; further each search engine would have different weights for various parts of the text, and it becomes a real differentiator. It could mean that blog-spammers need to start generating more 'human like' prose, but that builds a heck of a hurdle for them to overcome in terms of AI. There's a 'feel' to a real review, that is very different than a press release, which is what most advertising reads like. I believe the difference is quantifiable.

I may be an idiot, in some ways. If I were to implement it, and be successful at it, I could probably make a lot of money selling the tech. One answer to that is that I've got WAY too many similarly difficult projects I want to work on (my JBidwatcher app, open tuple and queuing servers, a wireless (phone) version of JBidwatcher, a working model of a better RSS subscription synchronization method across my many machines, and more that I'll never get to, but I write down in my little Moleskine notebooks). I have virtually no time (what with earning a living and all) and I just want SOMEONE (Technorati, Feedster, pubsub, even Microsoft!) to implement it, so I throw it out there for those who haven't thought of it yet.

Do some statistical analysis, some stemming and meaning interpretation (perhaps with the Open Cyc ontology project), build a 'profile' of the authors voice. I believe you'll find over many posts that it should converge. If it doesn't (or it does too fast!), it may not be a persons voice. (Other possibilities include multiple authors, generated content, stolen content, etc.) It's roughly equivalent to Bayesian spam filtering, but across a much larger content range. Heck, maybe a simple Bayesian filter would work, if you combine it with user-directed filtering where over a certain number of 'Spam!' scores would place a given blog entry into the filter.

It's harder with folks like Scoble's writitng, which is usually so terse, but even Scoble has a voice that you can extract from his writings.

It means that you have to stop treating some data as 'pure data', and start analyzing it as real human text. Similar to Google's original revelation that you can treat the web as a graph of links where incoming edges indicate relevance, this starts treating the pages themselves as sources of knowledge instead of just combinations of words. It means a little more processing per-document. Is this too much? Will it overwhelm the search engines with new processing power requirements? I would argue that in order to remain relevant in the face of active subversion by advertising agents, the search engines don't have a choice, especially the blog search engines, who are at least just starting out.

Best of luck; I envy whoever ends up working on it. A lot.

-- Morgan Schweers, CyberFOX!

Shoot the engineer and ship the code!

2005-07-02T05:07:08Z

Greetings,
One hallmark of the better companies that I've worked for is the occasional need to shoot the engineers and ship the product.

The idea is that the engineers, justifiably proud of their work, will keep finding one more thing that they need to fix, one more bug to work on, and one more feature. All of them asked for/seen by the users or found by QA, all of them useful, but not all of them 'above the fold' for diminishing returns. At some point, the need for users to get a new version outweighs the advantages of continued development towards the mythical 'zero defects'. Thus the revolver and the release.

I believe I've reached that point with JBidwatcher 0.9.7. I finally fixed the selection model, a long standing point of frustration for me. Afterwards, my thought process went like this...

The next thing is to make it recognize eBay's new Dutch format and put the number of items out of a dutch bid that you're winning in parenthesis. This leads to the thought that the sum of prices should include multiplying the quantity you're winning against the per-item price so far. This leads down a rathole of whether the prices sum should have another field which is just stuff YOU are winning... Beyond that rathole is another one, adding new fields to display as columns. In order to do that, I need to parse out the location, the seller feedback (and seller feedback %), and make sure all that works, and when it doesn't that it doesn't interfere with the fundamentally important features. There are also dozens of little UI touchups I want to do, and I'm not sure everything in the context menu is reflected in the normal menubar... Oh, and the updating code needs to be improved, so that I can point-disable certain features if they fail, so users don't have to keep updated with the 'status' of the program...

If I were to start down the path of doing all that, I wouldn't release a new version this weekend, and maybe not even next weekend; maybe never! I've wanted to do a release for 4 weekends now, and every weekend I say, 'Well, this other thing is still not quite right, and it looks unprofessional next to the rest of it...'

In the mean time, I have dozens of (maybe even a hundred) users clamoring for a new release, and perhaps a thousand more quietly suffering because 0.9.7pre2 has succumbed to a certain level of 'bit rot', as eBay's changes have slowly made it less and less useful. It's still good, but you have to do a bit of work, and keep up with my updates in order to know what can still be done. That's a bad user experience, imo.

So, I'm shooting the engineer (virtually, not really, of course), and shipping the product. The changes for 0.9.7 are massive. The 'pushed off' feature list for 0.9.8 is also big. I'll talk about that in a few days, I think.

Before I get to the features, I need to mention that one of the slightly controversial things that the new JBidwatcher will do is ask if you'll allow it to use the Affiliate program to bid on items. It should ask once, and only once. This is a shot in the dark, for me. I don't know if it'll work, I don't even know if they'll actually pay me, but I believe JBidwatcher is driving useful and valuable traffic to eBay. That said, there is a Decline button right there, and it works perfectly well. NO ONE should feel that they need to select 'Accept' for the affiliate program, for JBidwatcher to work. I explain more in the accompanying dialog. JBidwatcher is not going commercial, nor will it ever require the Affiliate program.

If the affiliate program falls through, I may try to package and sell JBidwatcher, but I will still have freely downloadable versions available. I'd be selling the ease of installation, the bundling of the JRE and the app (and of course the cost of media). There are those, especially in the Windows world, for whom having a physical copy is important, and getting a known-working Java runtime with it is useful.

Now, on to the feature set...

Features

Customizable Columns - You can right-click (or control-click on OS X) on the tabs, and completely customize what columns are shown for each tab, and the column customizations stick between sessions. There are several new columns to choose from, as well. New tabs initially inherit from the 'current' tab when created by searches.
Buy It Now - You can buy items directly from 'fixed price', or 'Buy It Now' listings. In conjunction, snipes and bids over BIN / Fixed Price values are rejected.
Subtract shipping from Snipes - A hugely requested feature, there is now a checkbox on snipes and multisnipes to subtract the shipping price from the snipe price.
RSS Feeds - a.k.a 'Gnomedex Mode' - You can 'subscribe' to several RSS feeds relating to data in JBidwatcher if you are running the internal web server. Those feeds can also be made accessible without logging in, for feed readers who don't handle authentication well.
Title-only searches are now supported in Search Manager.
Looks more like a native Windows executable for Windows. (Still requires the JRE, but has its own icon embedded, and much easier to download and run, and sets a better memory limit.)
Complete revamp of the Search creation and editing dialogs, including the ability to have multiple searches targetting a single tab, and better naming support. (You no longer have to create, then rename, a search.)
There is now a nice display at the bottom of the screen, showing how much the sum total of all the selected auctions prices are, with approximations in USD if multiple currencies are selected, and shipping.
Many more message boxes have 'Don't show me this again' checkboxes.
The context menu has been shrunk, and reacts in a context-sensitive way to what you have selected.
Now detects most bids placed outside of JBidwatcher, and displays them.
Shipping, current and max bid are all sortable, no matter how many different currencies are represented.
If logging in to the user account failed, 'Not logged in' shows up in the upper right, instead of the eBay username and eBay time.
You can turn off JBidwatcher's 'deleted items' tracking in the General tab of the Configuration Manager. This is so you can run a search and have all the items loaded, even if you deleted them in the past. I recommend using this option sparingly.
Parsing code is a bit more robust in the face of unrecognized data, falling back to old values.
Many advanced (i.e. hidden) configuration options which will need to be documented.

Bug Fixes

Selection is preserved through delete events, add events, and resorting. Finally!
Reordering columns works, and is preserved between sessions, along with sort column and direction.
Multisnipes are no longer cancelled when you're the high bidder, but below the reserve.
If you ever turned on the proxy, but then turned it off, JBidwatcher still tried to use it, causing massive delays in almost all network activity! Fixed.
Should be much more resiliant in the face of network failures.
Ignores eBay's new 'blank' thumbnails.
Adult login works again.
Most item changes show up instantly, no moving the selection around to see them.
Does not hang when loading invalid searches.xml files.
The HTML view uses relative paths where possible.
Removed 'File | Exit' on the Mac.
Show Information' converts to a scrollable dialog if there are too many 'events'.
Searches can load hundreds of items at a time, again.
You can delete comments.
Initially all updates, snipes, and bids are put off until the first sign-in succeeds.
Delete now initially highlights 'OK', instead of requiring you to to tab to it.
Best Offer auctions don't show up as 'null' anymore, but are not yet supported.
Deleting an auction also removes the data that JBidwatcher keeps on disk related to that auction.
30-day 'renew if unsold' store listings should automatically be recognized as 'alive' again, after completion.
Some failures to start or close due to odd characters in titles should be reduced, and maybe eliminated.
Recognize 'start time' and 'initial bid' prices, which we haven't been doing for months.

Known Issues

Some RSS readers will fail on the feeds provided when high-bit characters are in the eBay title.
Does not show the number of dutch items you are currently winning.
Does not take into account the multiple of 'won items' times 'dutch price' when calculating the price total.
Many requested columns are not yet implemented.
The snipe dialog sometimes gets into a bad state where it is mis-sized and 'OK/Cancel' are invisible, or the text input boxes are oversized.
Multiple monitor setups are still probably not working well, as I can't afford a dual-monitor setup to test on yet.
Often-repeating searches may cause memory issues over a long period of time on Windows. This should be reduced in this version if you're running the .exe version, but it probably still exists.
Still no HTTPS proxy configuration in-program.
FAQ only got a few changes, and minor additions, it needs a lot more changes.
Cmd-I is still 'Add Auction' on OS X, although it's been pointed out that it should show information. I just don't know what to replace it with.
RSS feeds won't work if the internal web server is disabled, but the checkboxes don't dance the 'disabled' dance properly, so you can have RSS feeds on, but the web server off, and it just won't work.
The program has gotten minimal testing with Java 1.5, so if you're running 1.5 and notice anything odd, it's doubly-important to report it.

While JBidwatcher always is, this version in particular has been a labor of love, and I'm thrilled to be able to share the intense hard work that went into it with the world. I think it's one of my best, cleanest versions ever, and I hope that when it lands in the hands of a few thousand users, you'll agree. (That is, I hope nothing goes wrong. :) )

The official release will be up on Sourceforge in a few hours.

-- Morgan Schweers, CyberFOX!

Exhausted...

2005-06-28T00:28:36Z

Greetings,
Wow. I'm totally exhausted after my weekend. I attended Gnomedex, hung out with Aryeh Goretsky (an ex-coworker and good friend), sat behind Robert Scoble on one of the days, pimped JBidwatcher on the famous Red Couch (wish I had a picture, I was SO stage-frightened).

Gnomedex was wonderful, happy, geeky goodness. For example, around 30 seconds of talking with the JotSpot guys reminded me why I love tech so much, I just locked into the understanding of how they were doing the 'live Wiki updates' instantly and thought it was SO(!) cool, and a great cross-platform browser-based app. Far more interesting than what I'm doing, in general.

Interleaved with that, I spent time with my fiance and her mother and sister (i.e. my future mother-in-law and sister-in-law) who were up to help my lady love shop for her wedding dress, and things went a LOT better than we were afraid they would. That all said, it was an...intense weekend, extremely solid separation of 'geek time' and 'family time', since her family is very not aware of technology. Further I had to be on my best social behavior, since we WERE with my future mother-in-law.

So I'm exhausted.

I need to clean up the new RSS back-end interface for JBidwatcher, so there'll be an RSS button that pops up a dialog with a bunch of buttons you can click to copy-to-clipboard the RSS feed locations, so you can subscribe to various different kinds of 'event lists' within JBidwatcher.

I need to move the affiliate opt-in screen to the first time the program is run AFTER the time that they have to fill out the configuration information, so that users can run it and get a feel for it before being asked to make a choice about whether to opt into letting JBidwatcher use the affiliate program. (And maybe I should run the poll on affiliate use.)

I need to track down some reported bugs with the new Search Manager code, and fix the saveAuctions bug (not closing, or successfully saving) where an auction contains a two-byte character that the brutally-simple XML encoding code I have doesn't handle right.

I need to do documentation. Lots of it. (Update the FAQ and re-word some of the questions, Update the 'Colors and Icons' doc, Add Linux launching to the Quick Start guide. Maybe put the LGPL in the Help menu. Perhaps offer a 'What to do?' button on intial configuration, to explain the very basics.)

I need to seriously consider moving the homepage off of Sourceforge, so I can build one-click downloading into the JBidwatcher home-page. This means surrendering my competitive urges to fight it out for top-100 Sourceforge project status. I get so intensely proud of 'being in the top 100', or 50, or..., that I forget that the download process is intensely ugly, and a potential real limitation to users getting the program. However, my own server bandwidth may be another equal limitation, as I have around 25,000 downloads of the most recent version and in my own hosting, I'd have a max of around 56,000/month; a factor of 2 is not much hedge. :) This won't happen in the 0.9.7 timeframe, but maybe by 0.9.8, or the rapidly approaching (imo) 1.0.

If I can make money by the Affiliate program, I'll switch to a hosted service much quicker, of course.

I think that's really all I need to do to release 0.9.7, is make the RSS back-end more accessible, touch up the affiliate program request, fix a few bugs, and do some more documentation.

I'll try to get it released this week, but I'd love to have a weekend after my weekend. :)

-- Morgan Schweers, CyberFOX!

Another pre-release for readers of the blog...

2005-06-21T18:05:46Z

Greetings,
This is an update to the beta versions I'm putting out there for people to try. All the caveats listed in the post 'A bonus for those who read my coding blog...' still apply.

PLEASE give me feedback on the general quality level, and any issues you run into with this version.

The specific files are:

OSX - http://www.vixen.com/beta/JBidWatcher-0.9.7-pre4.app.tar.gz
Windows - http://www.vixen.com/beta/JBidWatcher-0.9.7-pre4.exe
Linux/Other - http://www.vixen.com/beta/JBidWatcher-0.9.7-pre4.jar
Source - http://www.vixen.com/beta/jbidwatcher-0.9.7-pre4.tar.gz

Thank you very much, and I hope JBidwatcher works well for you!

-- Morgan Schweers, CyberFOX!

A bonus for those who read my coding blog...

2005-06-04T10:37:21Z

Greetings,
I am stating up front, that while I always disclaim any suitability for anything, the current beta is EXTREMELY likely to be unstable, because I've changed thousands of lines of code since the last release.

One major new feature is support for completely custom columns. Right-click or control-click (on the Mac) on the tab name to customize them. The UI for the tabs isn't complete, I've still got a bit of work to do, and I'm not documenting all that's changed here. (I need to get to sleep, I leave for vacation at noon tomorrow.)

Suffice it to say, I feel the program has been massively improved.

There are a few features I still need to add before I release 0.9.7:
- Make 'New Search' include a 'destination tab' & widen it.
- Update the FAQ as much as possible.
- Update the 'What do the colors mean?' document.
- Reduce menu options when clicking where tab isn't.
- Configure default 'display.cfg' to do the right things.

These would be nice:
- The tab context menu deleters should be submenus.
- Set focus on 'Yes' when deleting items.

That said, as a benefit for those who follow my blog, I'm telling you where the beta is. It is very probably NOT ready for prime-time, in my opinion, and I very seriously want feedback on how it works for you, what you think of the new features, and any other thoughts you might have. Please don't pass it around, or tell others the URLs.

The URL is NOT on the www.jbidwatcher site, this time, it's on a site I run, specifically because I don't get traffic logs from www.jbidwatcher.com, and I do get traffic logs from my own site, and I want to know how many people download this beta.

The specific files are:

OSX - http://www.vixen.com/beta/JBidWatcher-0.9.7pre3.app.tar.gz
Windows - http://www.vixen.com/beta/JBidWatcher-0.9.7pre3.exe
Linux/Other - http://www.vixen.com/beta/JBidWatcher-0.9.7pre3.jar
Source - http://www.vixen.com/beta/jbidwatcher-0.9.7pre3.tar.gz

I'll be on vacation until Tuesday evening, but please, please feel free to fill my cyberfox .at. gmail.com inbox with feedback!

-- Morgan Schweers, CyberFOX!

Wow...

2005-06-03T18:25:12Z

Greetings,
Last night, in a fit of frustration/genius/coding that resulted in me getting only 5 hours of sleep, I completely revamped the column control code and added customizable columns into the program.

Don't want to see 'Max' or 'Shipping' on your Selling tab? Remove 'em!

Want to have Comments as their own column on Current and any custom tabs, but not on Selling? No problem!

Don't like the 'countdown' approach, and would rather just see 'End Date' in all the tabs, or even just a few of the tabs? No problem!

Want to split out snipe vs. max bid into two columns, or Shipping and Insurance into their own columns? All doable.

In the end, I'll probably have about 20 custom columns that you can pick and choose from for each tab that you control, and their information will be saved (including order!) between runs.

This should clear out about 5 bugs and probably 20-30 feature requests. :)

...but damn, I need my sleep. Ah well, off to work with me now!

-- Morgan Schweers, CyberFOX!

p.s. Remember, disable 'Registered Adult' in the eBay tab under JBidwatcher's Configuration Manager to fix the current round of sniping problems.