This is informal wishlist of things people would like to see improved in Webware. It complements the ToDo list. Items that are completed or moved onto the ToDo list should be removed from this list (copy and paste the text to the WishesGranted page). Please sign and date your new wishes and comments on existing wishes. Use horizontal rules to separate wishes.
This page is for extensions to the Webware core. Autonomous items like Kits belong on the AppIdeas page.
Contents
Note: I've been working on an experimental refactoring (WebwareExpRefactoring) of the Webware code that addresses many of the issues/wishes listed below. It is not a fork, but is also not 'official' webware code yet. Help is needed testing it and assessing if it is ready for 'official' review by ChuckEsterbrook. Please follow see WebwareExpRefactoring for information on how you can help out. -- TavisRudd - 14 Jan 2002
These are a few objects/applications we should evaluate someday and see if they are worth porting to Webware.
Horde -- (PHP) contains classes for dealing with preferences, compression, browser detection, connection tracking, MIME handling, and more."
IMP -- Horde webmail client. IMAP/POP.
-- MikeOrr - 04 Apr 2002
Currently, ExtraPathInfo is a site-wide setting in Configs/Application.py. But it should really be set on a per-servlet basis. The servlet knows best whether it was designed to use ExtraPathInfo, and it's usually only a few servlets per application that do. ExtraPathInfo can cause unexpected behavior with servlets that are not specifically designed for it; e.g., see ExtraPathInfo.
I propose a new HTTPServlet method, .wantExtraPathInfo_(), returning false. Servlets or intermediate classes can override it if they want to provide this feature. Thus, the first servlet found determines the setting. If it's false and there is extra path info, return a 404 Not Found error, because we've reached the wrong servlet (perhaps an index.py WebKit implicitly inserted into the URL).
-- MikeOrr - 03 Jan 2002
It's been a planned feature of Webware for some time, but hasn't actually been there: for Webware to use locally-developed subclasses of major classes, like Application, HTTPRequest, etc. Not only would this help make Webware easier to upgrade -- since any local enhancements would be clearly seperated -- but it would also make experimentation much easier. Non-core-Webware-developers could easily make and distribute enhancements, and even core developers might appreciate being able to distrubute controversial enhancements.
As these enhancements get wider use, we could see how they work in real situations with a greater variety of developers (and styles). Some will probably fall by the wayside, but others might show gretter than expected popularity.
-- IanBicking - 03 Dec 2001
In the meantime: On Tuesday 11 December 2001 12:28, Geoffrey Talvola
Chuck has written some nifty code to make it easy to replace class methods after the fact. It's in MiscUtils/MixIn.py. You can put an __init__.py file into your context directory. This gets imported when the appserver starts up, and is therefore a convenient place to put in code that needs to modify the core Webware classes. Something like this should work as the contents of your __init__.py:
###############################################
from MiscUtils.MixIn import MixIn
from WebKit.Session import Session
class SessionMixIn_:
def expiring(self):
# your implementation goes here
# Now inject the methods from SessionMixIn into Session
MixIn(Session, SessionMixin)
###############################################
Until Webware has a better way to support custom classes, this is probably the cleanest way to get what you want.
-- TavisRudd - 11 Dec 2001
self.request().field() (in HTTPRequest.py) currently returns either a string or a list. However, it causes major inconvenience. If there is only one value for the field, it's a string. If there are multiple values, it's a list of strings. This behavior is inherited from Python's cgi.py module. However, it's inconvenient for servlet writers. It requires the careful programmer to have to check the type of _every_ field, even those that should always be one or the other, or risk having the servlet crash if unexpected input is received.
How about adding a .fieldMulti() method that always returns a list? A single value would be coerced to [value], and a missing value to []. This would allow servlet writers to skip the list test.
Additionally, how about changing .field() and .fields() to never return a list. Instead, choose one of the values arbitrarily and return it as a string. This would also allow servlet writers to skip the list test, in cases where they want only one value and don't care which one it is.
If changing .field/s() as above is too backwards-incompatable a change, how about leaving .field/s() alone and adding .fieldSingle() with this behavior?
-- MikeOrr - 30 Nov 2001
Consider proposing improvements in multiple value handling to Andrew Kuchling for Python's cgi module. Then other projects can get the benefit of such improvements. Of course, Webware will still have its own distinct approach for accessing field data, but we would at least be able to take advantage of common work done in this area.
-- PaulBoddie - 03 Dec 2001
This isn't a substantial change, but rather a cleanup of semantics in the interface. IMHO, the interface in cgi doesn't have particularly clean semantics anyway, and tacking yet another thing onto it won't help, and will just introduce yet another backwards compatibility issue.
Personally, while I always use a function to make multi-select-fields into lists, it's almost always a bug if I get a list when I didn't expect it, and I'd rather have it raise an error than throw values away.
-- IanBicking - 03 Dec 2001
After making 2 enhancements to WebKit, then we will probably be able to debug our persistent web apps in a graphic environment like WingIDE.
Provide a SequentialAppServer that is not multi-threaded.
Provide for reloading of any modified modules prior to serving a request.
In fact, this will be my highest priority after the 0.6 release, as it could faciliate faster development of both Webware and the sites built with it.
-- ChuckEsterbrook - 03 Nov 2001
This should be fairly straightforward using the WebwareExpRefactoring code I've been working on. I've already factored all the threaded stuff out of the AppServerClass so all that we would need to do is write a simple socket connection dispatch class that doesn't use threads.
Is this even possible?? Are you thinking of this in terms of a non-persistent AppServer that works like OneShot?
-- TavisRudd - 05 Nov 2001
UPDATE: I've completed step 1 from above. SessionStore still has a worker thread for managing session expiries, but I think it shouldn't be a problem. I've done some more looking into step 2 and don't think it is possible without restarting the AppServer completely. Modules can be reloaded, but anything that has been imported from them will not be reloaded. In terms of debugging, a complete restart would make sense any way. -- TavisRudd - 23 Nov 2001
N/C -- TavisRudd - 31 Oct 2001
Both databases are as close as one can get to Open Source/Proprietary twins. They feature multi-versioning concurrency control, which means that readers never block writers and writers never block readers. Performance under concurrent access to data is far superior to databases that use table or row level locking. That both support transactions with rollback, triggers, etc. and are fully ACID compliant means great safety and an easier sale into corporate IT shops. Refer to http://www.zend.com/zend/art/databases.php for example benchmarks.
-- EdmundLian - 31 Oct 2001
Webware would be more immediately useful if it had out-of-box support for simultaneous access to established databases like Oracle, DB2, PostgreSQL, etc. using pooled, persistent connections. This would allow Webware to be used as a front-end to traditionally designed (as in non-object-oriented, non-MiddleKit-style) schema. I know there's a patch for this out there, but it does not seem to have made its way into the CVS, nor has it solidified into some kind of plug-in or kit.
-- EdmundLian - 1 Nov 2001
(MiscUtils.DBPool provides something like this -- IanBicking - 02 Nov 2001)
Sort of... to be truly useful, pooling needs to be part of Application.py as proposed by Ken LaLonde here. Some prefer it to be a SitePage method, so we need to figure out how to accomodate both camps. -- EdmundLian - 05 Nov 2001
Would it be possible to integrate other database pooling systems into Webware? Here's one I found on Parnassus! -- PaulBoddie - 08 Nov 2001
Another system to consider for integration with Webware is SQL Relay. From the website: SQL Relay is a persistent database connection pooling, proxying and load balancing system for Unix and Linux supporting ODBC, Oracle, MySQL, mSQL, PostgreSQL, Sybase, MS SQL Server, IBM DB2, Interbase, Lago and SQLite with APIs for C, C++, Perl, Perl-DBD, Python, Python-DB, Zope, PHP, Ruby, Ruby-DBD and Java, command line clients, a GUI configuration tool and extensive documentation. The APIs support advanced database operations such as bind variables, multi-row fetches, client side result set caching and suspended transactions. It is ideal for speeding up database-driven web-based applications, accessing databases from unsupported platforms, migrating between databases, distributing access to replicated databases and throttling database access.
-- DanielClark - 31 Jul 2002
There should be a standard module for doing browser detection. There's probably good PHP code to copy for this. It should have facilities for adding your own semantic browser-capability tests (for instance, you should be able to ask, "can the user's browser do X?" instead of just "what browser is it", and then you can add the test for what browser does what elsewhere (and grow that definition with increased testing)
-- IanBicking - 06 Nov 2001
(a more extended discussion of this issue as it arises from the WebwareStyleGuidelines. In the meantime, check out CopingWithTabs)
(extended ideas on enhancing Webware's URL decoding)
It would be nice if Webware could dynamically serve files over FTP, analagous to HTTP (i.e., fully dynamic). More powerful with
UrlDecoding (since FTP doesn't have GET variables). Medusa implements this protocol, so maybe some of the code can be ripped from there. -- IanBicking - 06 Nov 2001
Interface should look like FTP, since the interface is fairly equivalent. I think FTP is a better (simpler) place to start. -- IanBicking - 06 Nov 2001
A flexible and seamless framework for handling authentication and authorization is essential for all the ideas listed on the AppIdeas page. It will also benefit users creating their own applications and serve as the foundation for the WebwareUserManager application. See UserHandlingProj for more discussion.
It would be nice if you could cache servlet output. This should also include caching by parameters -- for instance, cache pages that have identicle GET variables, or add on session ID, or add on browser type, etc. Essentially the cache should be based on two items: the servlet URI and a key that would be generated by the Servlet (or ServletFactory?) -- that key could include whatever information is important.
Of course, with customizable times and the potential to expire these caches prematurely.
This is also a feature request for Cheetah , and for caching of parts of a page it would be essential. However, caching the entire output should be well suited to Webware itself. If this caching mechanism was generalized and exposed, servlets could cache portions in an ad hoc manner as well.
-- IanBicking - 27 Nov 2001
Webware should compress non-streamed output (and maybe streamed output) with gzip when possible (which is at least 95% of the time with current browsers). This shouldn't compress already-compressed formats, like most image files. This is a large saving for most sites where bandwidth is at a greater premium than processing power. The implementation of this should be quite easy.
-- IanBicking - 27 Nov 2001
This result is easily achieved outside of WebWare by using Apache's mod_deflate module. This is very quick to install, Google it to find directions. This presumes you're using Python Server Pages / an AppServer. -- Kevin J. Rice justanyone - 4 October 2006
My favorite proposal to improve Webware's scalability is:
Allow multiple app servers to run on different machines. Each of these app servers would be given a unique "instance name" specified in a config file. Session IDs would have the instance name encoded into the session ID, for example an instance called Instance1 would create session IDs such as "Instance1_433048934".
Add options for the adapters to distribute requests to multiple appserver back-ends. Requests without a session ID would be distributed randomly (or in round-robin fashion) to the different appservers. Requests with a session ID would be directed to the particular appserver instance that originated the session, as determined by extracting the instance name from the session ID. This would be fairly easy to implement in the Python-based adapters -- i.e. the CGI adapter and the mod_python adapter. If the strategy proves useful with those adapters, then it could be extended into the mod_webkit adapter which would require more work.
The advantage of this strategy is that all session data is localized to a particular appserver. This is potentially much more scalable than having to dedicate a central database server to store session data.
Now you can have as many front-end web servers as you want, load-balanced using any off-the-shelf load balancing solution. And you can have as many back-end Webware appservers as you need.
-- GeoffTalvola - 03 Dec 2001
UPDATE: In release 0.7 I've implemented a setting SessionPrefix which can be used to add a machine-specific prefix to the session ID. This can potentially be combined with some mod_rewrite magic to achieve load balancing with session affinity in a slightly different way than I've described above, but so far it's just an idea; I haven't tried it. When I have the time I'll actually test the technique and write up a description of how to do it. If anyone's curious to try out the technique in the meantime, let me know and I'll try to describe what I had in mind.
-- GeoffTalvola - 14 Mar 2002
CGI adapters capable of opening and pooling socket connections back to the application server should do so. Currently the WebKit.cgi, WKCgi.exe, and the ISAPI version open a new socket for each connection.
In the case where the webserver is outside of the firewall and the application server is inside this will give a measurable performance increase.
-- AaronHeld - 12 Jan 2002
So I read about this WebWare thing. Looks cool. So I download the tarball, read the README, run the install.py, and expect magic to happen. Hmm, nothing. Okay, so I skim the online docs. I don't see it until my second read of the Overview: Most people start with WebKit, which is the Webware application server A few clicks later and I finally land in the install guide for WebKit.
My suggestion is that, coming into this in complete ignorance, you might want to provide a RoadMap straight up, front and center. As in: install the thing, run the app server, which web server do you have? for web server foo, do this, and go!
Just a little nag, and maybe it's just me. I like to see the thing go before I decide it's worth spending my time delving into docs (which admittedly wasn't long, but I'm a fast reader), and as simple as it finally was for me to get the thing running, it should be in my face from the get-go.
-- LesOrchard - 25 Jan 2002
I want to be able to trap when a browser cancels a transaction, without having to dig into ThreadedAppServer.TASASStreamOut and otherwise hack around :)
Basically, I have (potentially) long-running processes that acquire an RLock, but I need to release that RLock if the process cancels. Right now, I have no way to interrupt that process... the servlet continues chugging in the background, even though the browser's stopped listening.
I've seen the issue addressed in a mailing list posting, but couldn't find anything more concrete than Geoff's suggestion about trapping the EPIPE exception in TASASStreamOut.flush.
-- TrippLilley - 01 Mar 2002
MiddleKit's deletion code currently is very slow in the way that it checks for other objects that point to an object being deleted. Basically, it pulls in entire tables instead of doing smarter SQL SELECTs to bring in only rows that point to the object being deleted.
-- GeoffTalvola - 04 Mar 2002
PSP currently only allows you to define a single method in a PSP file using PSP syntax. If you need to define additional methods, you have to use the psp:method tag which only supports Python syntax. It would be nice if you could define multiple methods using PSP syntax.
There is a hackish way to do this using Python 2.1's nested scopes, but it is a bit limited. See MethodsInPSPSyntax for details.
-- GeoffTalvola - 14 Mar 2002
When you install Webware with "python install.py" it automatically generates some class hierarchy documentation and some summary documentation of all the classes. But it doesn't include enough information to be truly useful -- most notably, it doesn't include docstrings. We should improve this, perhaps by using pydoc in the install script.
It was reported on the Webware mailing list that pydoc doesn't actually work on Webware's source files for some reason.
-- GeoffTalvola - 10 Apr 2002
A serverSidePath_() function that works during servlet construction (i.e., in __init__).
-- EdmundLian - 12 Sep 2002
Packaging enhancements would be very welcome, especially for those of us who would love to see a Debian package.
-- ChadWalstrom - 6 Jan 2005
There are numerous advantages to using distutils:
users don't need to maintain a local copy of all the modules.
the core modules can be used easily by other frameworks
it can be packaged for Debian and other OS distros.
installs on Windows now make use of the GUI installer.
distutils offers many advanced packaging facilities!
distutils is the Pythonic way to do installs.
-- TavisRudd - 31 Oct 2001