I don't think IBM will update the NSF to "current standards". It's probably too big of a task.

Not sure how they will solve the problem. The easiest is probably to let the developer choose between NSF and a JDBC data source. Then they don't need to provide tools for migration from one platform to another.

personally I think moving to WebSphere is not so bad way. Look at Domino market, it decrease from Q to Q. So if IBM push us to WebSphere and do some way to move app from Domino to it, why is it bad?

Because Websphere is an overengineered piece of big and complex software.

Domino is so great because it is relatively simple and can be installed on one single machine without any headache.

Websphere is more like Microsoft stuff: you need websphere, a database server and this and that.

The NoSQL hype stops when vendors try to charge for it. Commercially NoSQL is unsuccessful in many cases. In addition nsf can handle Master-Master replication while most NoSQL solutions can't (the popular MongoDb for example).

For your topic I will keep the optimistic spirit and not say much. Personally I am not in a position to base my career on hope and vague statements but if IBM does something great I'll take a look for sure.

I hate this one!!! I would love for FT Indexes to be updated right away!!! IBM/Lotus has always skirted this one for 10 years...

I Observe and test from many time the solution NoSQL CouchDb

{ Link }

This is the way for me for very modern Web Application

CouchDb have the replication feature so that in Notes/Domino.

And today the development team go into Mobile Devide (Ios, Android...in this moment this is a theme very hot!)

The creator is Damien Kiatz ...someone remember this name?

(The people that in R6 of Notes re-writed the code for @Formulas language in C/C++ from Assembly)

{ Link }

The strategy of IBM I don't know....I think that IBM is a elephant to move...so that WebSphere is a elephant.

I think that IBM are not so clever to exploit NoSQL :-(

I think that NSF die in the coming years...but this is my thought.

IBM does use NoSQL. For example they use Apache Hadoop for InfoSphere BigInsights, see { Link }

They also paid Damien Katz developing CouchDb for quite a while until he succeeded gaining venture capital and start his own company (which does not even exist in this form anymore as they merged with another venture capital driven company).

IBM is a tech company. When they want something they can be very fast. And if they think they are not fast enough they can just buy a company that is.

I believe NSF will be around for a long time, whether IBM does updates the NSF format for larger data format, I do not know nor will I speculate. But that does not matter to me. Domino for us is just another data source and container and despite what other say a very good one especially how old it is. I have looked into using Couchdb, but it is not mature enough for enterprise level applications, but it will. I see the problem is how developers are using the NSF format. It should be treated only as a data container and you need to totally separate the visual part of the application with the data. I see this as a major problem with XPages. Don't get me wrong, XPages is a major improvement compared to creating classic Notes applications. For 95% of the companies in the world, the Domino infrastructure as it stands more than fulfill their needs. I wish IBM just fixes the bugs.

Julian,

The issues you listed are really important in showing the scalability of the Domino platform to the customers. Customers raise the eyebrows when we say, we cannot do JOIN in domino. The 'other' development teams in the department also make it a weapon against domino. For all of the developer communities outside Domino, join is a very basic capability of a database system. I rarely encounter any 32/64 bit issues(and I do develop some pretty big and complex systems). As Richard Moy said, 90% of the applications can be developed without any problems with NSF as it stands now, but if IBM can solve the 4 issues mentioned above, that will be a huge blessing for the developer community and also a stronger selling point for Domino. The capabilities IBM added by the introduction of xPages are really great, but adding capabilities like join, dynamic views and faster indexes would make it a killer platform.

For the record there has recently been some work by IBM concerning the 64GB limit, see

https://www-304.ibm.com/support/docview.wss?uid=swg21102305

(Credits go to Simon Ö'Doherty)

For the JOINS part. There are some concepts with XPages, see for example this blog { Link }

This technote doesn't say that I can have more than 64gb data.

@10 Henning,

We do that type of joins in xPages all the time. It sure helps to solve some issues. But a real join capability can do much more, especially when it comes to reporting.

Well the technote says that you can open a database that grew to more than 64GB in size which did not work in the past. And a database with more than 64GB of data has more than 64GB of data ;-)

To be clear I did not say that it has been removed just that some work has been done in this area. I just posted this for people who haven't read it before. I am not affected as I normally only do maintenance work for IBM (Lotus) software.

"If such a thing happens I will leave the IBM world at the first chance. ".... Would that not be like a Java programmer moving to another development platform because their company migrated from SQL Server to Oracle? If you build applications and solutions I am not sure the DBMS is terribly important. I like the NSF but NSF is just a small part of why I do Notes development. I like the whole concept of XPages and being able to do NSF development if my app requirements meet the needs of NSF and a SQL engine for those where it doesn't. The same Xpage applications should largely run the same on either.

Isn't the future of NSF called NSF2DB2? Uhm, I guess not... Flashback: "Do these guys know what each other is doing?" (And XPages? Just old JSP based technology wrapped in something else than "Garnet" originally was)

@Peter: it's not about changing the database system, it's about destroying a great platform for political reasons. We all remember the time where Websphere was planned to be the new Domino, and where IBM tried to convince Domino customers to migrate to Websphere, don't we?

The plan failed miserably, because Websphere is Websphere (huge, complicated, for big organizsations) and Domino is Domino (relatively simple, only needs minimal administration work, fits on one hardware etc).

I simply have the feeling that there are a lot of people inside IBM who still want to get rid of Domino in favor of Websphere, and since I cannot imagine any rational reason for that, that could be driven by polical reasons only.

And just if - and only if - THAT happens, that Domino dies because of political games, THEN I have no interest in that game anymore.

Our experience is that the most of our customers have a speed problem related to NSF after 2 years of using Notes. Even for small companies databases grow fast and users complain about slow views, searches etc. NSF is a real bottleneck for us in Notes in many areas. We do LN development for more that 15 years and there are no sifnificant changes in NSF ... Its really strange how IBM stopped developing this product. And I cant believe that there are no ways how, for example, increase the speed for LN views or FTsearch. And I can see some probably not so difucult sollutions like using Apache Lucene as an database indexer . These guys { Link } did it for LN mail search and its blazingly fast. Why not to integrate it into Domino server with NSF -> Lucene synchro. This could replace Notes view searches with ad-hoc queries, FT searches etc. But it seems to me, that IBM have no developers :-( ... as stated in article, NSF is oldest no-sql , but they stopped in 90's . During the last years we got MongoDB, CouchDB and many others that can make ad-hoc queries, can handle hundreds gigabytes of data with great speed etc.

In 2002 Steve Mills (who still runs IBM software today) said:

After this year's R6 deliverable, the next version will be a WebSphere-based version. Notes is 80 percent middleware today. It already has some WebSphere and pieces of DB2 embedded in it today. But it's fundamentally built on the Notes file system, which is a late '80s design point.

Source { Link }

That was 9 years ago and it did not happen. So maybe there have been a lot of people inside IBM who did not want to get rid of Domino?

For me the problem is that Mills probably won't put much money in this area until the technology hasn't finally changed. His vision of super cheap development based on Websphere and DB2 hasn't changed.

by the way if done right a Websphere transition could also mean that the product runs on JBoss or even a lightweight container like Tomcat,netty too using multiple SQL or NoSQL datastores. Not that I believe that this is going to happen but in theory it would be possible.

@Dave.

Couchdb has some wonderful features. However the following two are a real impediment as a serious NSF replacement in many many scenarios.

Couchdb has no readers/author fields AFAIK. Likely because it wouldn't be as performant. But you lose a significant feature to get that. Again AFAIK they suggest you have "personal" db's and then aggregate/replicate data as necessary. A somewhat different use case than the one reader/author fields solves.

Couchdb doesn't seem to allow you to protect your design if you want the single container model of the NSF. I saw some *initital* thinking about keys and certificates etc but it seemed to be a long way off.

@Julian

Point 1 should be fixed.

Point 2 is now less of an issue with DAOS. When combined with federated DB's (ie. Project per db etc) I can't see it being much of an issue for at least small and medium businesses.

Point 3 can probably be fixed fairly cheaply with resources.

Point 4 not sure *exactly* what that means.

Point 5 As you know you've ALWAYS been able to store data relationally in Notes and now with Xpages you can finally visualize it in the UI as well. Id' rather see mobile/Xpages investment than more fundamental joins etc.

P.S. For the first time in about 3 years I just opened a Project management tool I built for a software development process. It was only every used on Windows built with Notes 4 thru 7. It runs flawlessly on my Mac with 8.5 Eclipse version of Notes. Incredibly impressed. Everything just works. And it's quite a sophisticated app. The benefits of that pale in comparison to most everything else.

@Stephen Hood

You are completely right, I dont mean CouchDB or MongoDB as replacements, just as a reference that some small private companies can develop something from the ground ... but IBM NSF development stopped in 90's. NSF is much mature comparing features to CouchDB or MongoDB. Its the reason that make me sad that IBM is not developing NSF anymore.

@Dave.

Yeah I was just partly highlighting that some of those achievements came at the loss of functionality.

Maturity is part of the reason the development has slowed although I don't think it stopped. There were all sorts of db properties added since the 90's for example. I think it's fair to say that without fixes to some key areas the kind of improvements you would like to see won't happen - so in that sense it has "stopped". The NFStoDB2 and Eclipse initiatives have probably soaked up the majority of resources in the last 5+.

P.S. Damien didn't *exactly* develop something from the ground up :). He had a running head start on the concept. He certainly worked his a** off though to re-imagine the Notes/NoSQL model for an http world.

Cheers.

As a Notes/Domino Design Partner, I hear this "no work done on NSF since the 90s" line pretty often. And I have to point out that it's pretty narrow-minded and insulting. There have been many advances in the Notes Storage Facility since 2000. We've got transaction logging, DAOS, replication of unread marks, responded flags, design compression, data compression, attachment compression, improved fixup, streaming replication, the Eclipse VFS, and soft deletions.

While that may not have kept up with every system that had no install base in the last 11 years, it's a pretty impressive list when you consider the perfect track record of backward compatibility in that time. Sure, DB2NSF was a major distraction and pointless exercise, but in the end that cost about two years out of 11, which isn't a bad ratio.

That being said, I absolutely agree with Julian's list of things that should be addressed in the platform. And I'd add "transaction bounds on operations" and "NSF-based events like onConflict or onUpdate."

In fact, I think both of these are more important than fixing 64GB or 16K limits on anything (if you're writing applications that need more than 64GB of local storage and 16K of summary information in a document, your architecture is bad. Fix it. You'd have a problem on RDBMS systems anyway.)

But there's nothing served but one's own chest-thumping to say "IBM hasn't advanced this since the 90s." They have. Maybe not in the way you want, but that's different than not-at-all.

@Julian if IBM made your XPages code from a .ntf run on a Websphere server, why would you consider that a "political decision" that you have a problem with? This is like saying that adding a 64-bit version of Domino for Windows is a political decision. Okay, technically it's driven by policy, so it's political, but that doesn't automatically make it bad.

Making the Domino server a WAS platform isn't inherently a bad thing. Doing it without the straightforward simplicity of the Domino setup is a bad thing.

Nathan, your last sentence is the whole point. If - again, this is all pure theory - IBM would move XPages to Websphere, then it will not be with the simplicity of Domino, I'm pretty sure of that.

It's like an attempt to make a Post-Panamax-Tanker act like single handed sports yacht.

And I have no problem with making XPages available on Websphere, I would have a problem with doing that in order to destroy Domino. Because there aren't only the big enterprises in the world, there are lots and lots of small and medium businesses out there, which I like to have as customers.

And the most of them won't even think about installing Websphere as a replacement for Domino. They would switch to another platform.

But, that has been discussed already some years ago, so let's not start it again :-)