1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Making new library -- "Not a framework"

Discussion in 'JavaScript' started by deathshadow, Mar 20, 2014.

  1. deathshadow

    deathshadow Acclaimed Member

    Messages:
    9,510
    Likes Received:
    1,929
    Best Answers:
    247
    Trophy Points:
    515
    #21
    Oh and I'm hoping to buckle down and get the last of the documentation written and live online over the next 48 hours, along with the first beta. Thing is as I write docs I keep thinking on new and better ways of handling things.

    One big change was the '_.extend' method that adds new methods to existing DOMElements. The way I was doing it I was passing the method to be attached to a function that checked if it existed -- the overhead of parsing that function and sending it on every _extend was taking too long -- of course it was dumbass! I know better than that, and I went and did it anyways.

    I've moved them into an Array type property of _, called _.extensions. The array is called for the name of the function first to test if it exists, the function only being accessed when actually added. This results in a direct copy instead of a stack operation, lowering the memory use and speeding up how fast it can be done GREATLY. (FF and IE REALLY seems to like this change in methodology... Chrome just kinda went "meh"; not really a surprise there.)

    Since it was in an array, I added a new method _.extensionAdd that handles adding new methods to the array, as well as checking/adding them to _d, _d.head and _d.body; so those three core elements and any newly created/selected elements will have those auto-attached. Care will be needed in terms of load order if other 'extensions' to the library are going to be used. Pretty much anything that uses _.extensionAdd should be run before any code that requires those additions.

    Likewise I've moved the String/Array polyfills and new functions into similar properties of _. Did wonders for the libraries startup speed, particularly on modern browsers that don't need the polyfills.

    14.6k and holding... getting really close to that 16k limit now.
    SEMrush
    I'm also playing with the idea of making it a bit more ... modular, so you can pick and choose the parts you 'need' -- like not everyone using it needs the AJAX functionality... so _library.js be the core without ajax, _library.ajax.js as the one with? Starts to become an issue of what do I include in 'core' and what is treated as extensions -- still the ability to pick and choose what you need is something I think a LOT of the larger 'frameworks' could benefit from.

    Would also let me make 'stock' extensions like .forms for form validation and form trickery, above and beyond the limitations of my 'core' library. Problem with that is I don't want to fall into the same trap as that massively bloated idiotic train wreck of code known as jQuery.ui -- I'll have to tread lightly.
     
    deathshadow, Apr 2, 2014 IP
    SEMrush
  2. AbstractChaos

    AbstractChaos Member

    Messages:
    58
    Likes Received:
    4
    Best Answers:
    4
    Trophy Points:
    25
    #22
    Financial data reconciliation and integration platform. All source data files are unknown at compile time and the UI constructs screens using configuration tables with more configurable options than I care to think about and the number original mentioned includes the backend processing system to get the files, load them and run tasks.
     
    AbstractChaos, Apr 2, 2014 IP
  3. deathshadow

    deathshadow Acclaimed Member

    Messages:
    9,510
    Likes Received:
    1,929
    Best Answers:
    247
    Trophy Points:
    515
    #23
    So in other words what I was doing making double entry accounting systems in Borland Paradox 20 years ago that the pScripts and queries fit on a pair of 1.2 meg floppies. Gotcha. :D

    Though admittedly at the time we had a very consistent and common data format that everyone and everything used for sharing data, particularly anything financial... it was called CSV... We did have things simpler; NOBODY was using anything more complex than the bottom 7 bits of ASCII for example (A habit I still use when coding to this day... I use anything outside that I'm diving for the entities)

    Gah, I'm starting to sound like a Weird Al song...
     
    Last edited: Apr 2, 2014
    deathshadow, Apr 2, 2014 IP
  4. AbstractChaos

    AbstractChaos Member

    Messages:
    58
    Likes Received:
    4
    Best Answers:
    4
    Trophy Points:
    25
    #24
    I guess it a little like that and also trying to match up billions of rows of trade data across 1000s of files where they don't all share the same identifiers, are all in different data formats and it is usually requires cleaning the data. Have to pull files from sources that again can be configured by users in 100s of different ways.

    Unfortunately even though I can see your point I inherited this codebase and its size I have reduced it by half of what it was so far (without removing any "features" and i use that term loosely) and no time to start again so just have to make do.

    Anyway back on topic.
    I like the idea of a modular approach for js files, though maybe a note in the docs to ensure users combine the js files into one when using more than one file
    (I would suggest a File Generator similar to JQueryUI's but i know you don't like extra steps)
     
    AbstractChaos, Apr 2, 2014 IP
    deathshadow likes this.
  5. deathshadow

    deathshadow Acclaimed Member

    Messages:
    9,510
    Likes Received:
    1,929
    Best Answers:
    247
    Trophy Points:
    515
    #25
    Since the sub-modules would rely on the core, they would already be single files and I would NOT even HAVE links to standalones...

    Hmm... maybe a detection and alert if 'core' is already loaded? That way if someone tries to include different modules more than once it will bitch about it.

    In terms of making it 'easier' for people to choose and include the correct file, I'd probably just have a scripted series of checkboxes that builds the appropriate include name.

    But really, just making it so that the official 'modules' aren't available as single files in the first place would probably be simplest. Only risk then is people trying to include the same code more than once, and making it throw a wobbly over that with an alert would put a right proper stop to that. Maybe make the alert detect which versions are included and which one the developer should actually be using? That might throw it over the remaining 1.6k though... (but I'd gain some back by moving AJAX out of core)
     
    deathshadow, Apr 2, 2014 IP
  6. deathshadow

    deathshadow Acclaimed Member

    Messages:
    9,510
    Likes Received:
    1,929
    Best Answers:
    247
    Trophy Points:
    515
    #26
    Actually, the detection and 'what version you should be using' code turned out a lot smaller than I had thought... untested, but comes in at 541 bytes, a little over a quarter of that being the text in the alert.

    Don't think I'm splitting out AJAX, at least not just yet. It's barely 2k between the actual method and the serialize function... Which returns a false if type="file" is set and a value chosen, so any handler built to use it can fall-back to a normal submit if a file submit is involved. Should ease automation.
     
    deathshadow, Apr 2, 2014 IP
    PoPSiCLe likes this.
  7. deathshadow

    deathshadow Acclaimed Member

    Messages:
    9,510
    Likes Received:
    1,929
    Best Answers:
    247
    Trophy Points:
    515
    #27
    Just to whet the appetite, this is the demo code that will be in the documentation for _.ajax

    var
    	status = _.make('#xmlStatusSpan', 'Unsent', 0, _.make('#xmlStatusDiv', 'XML Status:', 0, _body)),
    	x = _.ajax({
    		state_1 : function(e) { status.textReplace('Opened'); },
    		state_2 : function(e) { status.textReplace('Headers Recieved'); },
    		state_3 : function(e) { status.textReplace('Loading'); }
    		status_404 : function(e) { status.textReplace('404, XMLHttpRequest Not Found'); },
    		status_200 : function(e) {
    			status.textReplace('Completed Ok!');
    			alert(e.responseText);
    		} 
    	});
    
    x.open('get', 'yourFile.txt', true);
    x.send();
    Code (markup):
    *NOTE* if you overload state_4 then status_ methods won't be called.
     
    deathshadow, Apr 3, 2014 IP
  8. AbstractChaos

    AbstractChaos Member

    Messages:
    58
    Likes Received:
    4
    Best Answers:
    4
    Trophy Points:
    25
    #28
    Thats looks very clean & simple :D. What is the boolean for below?
    x.open('get', 'yourFile.txt', true);
    HTML:
     
    AbstractChaos, Apr 3, 2014 IP
  9. deathshadow

    deathshadow Acclaimed Member

    Messages:
    9,510
    Likes Received:
    1,929
    Best Answers:
    247
    Trophy Points:
    515
    #29
    ... and that is what's wrong with how most people learn AJAX, they don't learn how it actually works thanks to libraries like jQuery.

    That boolean chooses sync (false) or async (true). Synchronous AJAX usually hangs, crashes or gives a "not responding" message which is why nobody actually uses it. The "open" command used is NOT part of my library, but is actually the native XMLHttpRequest object. MDN has good docs on that.

    https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest#open()

    It's optional by the standard, but the original ActiveX equivalent defaults to false instead of true, so it's safer to include it.

    Part of my intention with the library is not to hide how js 'really works' and instead just put in some functionality I consider to be missing. That's why -- for example -- the AJAX command returns an actual XMLHttpRequest object or it's activeX equivalents. I'm really against hiding how things 'really work' -- which seems to be the intention of frameworks.
     
    Last edited: Apr 3, 2014
    deathshadow, Apr 3, 2014 IP
  10. AbstractChaos

    AbstractChaos Member

    Messages:
    58
    Likes Received:
    4
    Best Answers:
    4
    Trophy Points:
    25
    #30
    Thought I recognized it, but wanted to check. :)
     
    AbstractChaos, Apr 3, 2014 IP
  11. deathshadow

    deathshadow Acclaimed Member

    Messages:
    9,510
    Likes Received:
    1,929
    Best Answers:
    247
    Trophy Points:
    515
    #31
    Side note, I seem to be saying "MDN has good docs on that" a lot lately. :D
     
    deathshadow, Apr 3, 2014 IP
  12. deathshadow

    deathshadow Acclaimed Member

    Messages:
    9,510
    Likes Received:
    1,929
    Best Answers:
    247
    Trophy Points:
    515
    #32
    Phew... been years since I wrote documentation for anything like this... last time would be my crappy little OpenGL raster font library.

    I JUST got everything except the actual DOMElement extensions documented... Which unfortunately is the halfway point on methods, but in terms of the web interface it means I've got things pretty well hammered down.

    Trying to decide what I want to do for comments... I'm thinking that for now I might resort to using either Disqus or IntenseDebates; I'm not wild about either of them but this is the first step on the road to my next-gen CMS, and I don't have the time to finish that right now; but since I'm violating my existing beliefs by even making this library in the first place, I might as well throw all my other "I wouldn't do that" out the window too.

    Might even use this as an experiment to see if advertising is the same asshat scam it's always been... I could probably squeeze a 160x600 in there.
     
    deathshadow, Apr 3, 2014 IP
  13. AbstractChaos

    AbstractChaos Member

    Messages:
    58
    Likes Received:
    4
    Best Answers:
    4
    Trophy Points:
    25
    #33
    Pray do tell :p

    Also I thought the library was more a set of helper methods rather than a "Framework"?
     
    AbstractChaos, Apr 3, 2014 IP
  14. deathshadow

    deathshadow Acclaimed Member

    Messages:
    9,510
    Likes Received:
    1,929
    Best Answers:
    247
    Trophy Points:
    515
    #34
    I originally wanted to finish it BEFORE I started on this project, but every time I get to user management I glaze over and lose interest. I wanted to host this js library on the new system, but for now I'm just going with a simple poor man's until I can buckle down and really dig into doing some hardcore PHP coding.

    Every few years I throw out the back-end I use for PHP/mySQL sites and start over from scratch -- usually because I find some better way of doing things or way of making things more secure... The current build would officially be revision 10... It's the ... pinnacle of my coding capabilities, but that's part of why it's taking so long to finish as I'm dotting every t and crossing every i... wait, that's not right.

    The highlights of that WIP CMS:
    1) PDO driven, extends the PDO object and restricts it's scope to things that need it. Quite literally it's only passed to objects that should have access to it, and is kept private and/or local in scope.

    2) Extends PDOStatement to allow locking, so you can pass statements to output/template code WITHOUT fear of it being reused to run another query with arbitrary values. One of the really cool things about PDOStatement is you can ->execute with a different dataset the same prepare; unfortunately that makes it insecure to pass it to things like the template. Most people (including myself until recently) get around this by building arrays of results, but that sucks on RAM hard and slows execution. Simply extending PDOStatement with a ::lock method that disables execute and can't be re-enabled means you can use it for data processing normally, and lock a result to pass it without fear. As a rule of thumb I only use multiple executes for UPDATE or INSERT, not SELECT. Since UPDATE and INSERT really aren't used for data you'd pass to the template...

    This is something I think PDOStatement should have had from day one.

    3) "one index to rule them all" -- ALL user requests are routed through a single index.php. You can probably search my posts and find me rambling about this time and time again. One entry point means one security gate to cover.

    4) UN/PW/Host/other DB connection info while included from an easy to edit file, is only made visible execution-wise to the main() function in index.php by checking debug_backtrace. Combine this with highly recommending that open_basedir be set to only allow access to non-code directories and it should block ANY possibility of leaking that information. (short of a FTP login)

    5) general login/user handling will be "getter only" to pretty much all execution.

    6) Implements "named queries" to make it easier to use PDO to access other database engines. Basically there are .ini files that contain queries, one file for each 'module' in the system (common, user, posts, etc) set into their own directories for each supported SQL type. (mysql, mssql, postgre, oracle, etc). In addition the db.php file containing the extended version of the PDO object is customized for each engine, providing an even more 'engine neutral' API at the CMS level.

    For example: (assumes $db is an extended version of PDO)
    $statement = $db->prepare('getUserIdByPassword','users');

    Would check to see if 'users.ini' is loaded from the appropriate location, if not it loads it to "private PDOExtended::queryList" as ['users'], then checks $this->queryList['users'] for 'getUserIdByPassword'. If present it prepares that query using the parent PDO::prepare. Since there are different db.php files for each engine, any special case handling can be customized for each engine. (an example of which would be pulling a result count in mssql, something it still exhales on the proverbial equine of short stature on doing...)

    7) Actually uses PHP for the templates... Since php IS a templating system. Running some idiotic halfwit 'templating system' like smarty on top of a language that is a templating system unto itself is just completely ****tarded IMHO. We'll have none of that in here. If someone is too stupid to use a PHP templating system with a few functions, they probably have NO ****** BUSINESS making templates in the first place! (honestly, I find it simpler to just use PHP than most templating crap people put on top of it anyways... one of those places where I end up screaming at the display "OH FOR *** SAKE, JUST LET ME USE THE DAMNED LANGUAGE!!!")

    8) uses anonymous functions (or in re-re speak, closures) to build two arrays of template commands, one array flat lists available methods, the other is an execution order that a simple template::run() command will parse through. It will have DOM style list walking and tree-type layout (firstChild, nextSibling, Parent, etc) as well as a 'getById' function so you can add/remove template modules where you want them to run. This should allow extensions and mod capabilities similar to systems like wordpress, without the 'blind include execution' spaghetti nonsense that opens up all those endless doorways for crackers to stick their foot into. Scripting 101, includes should NEVER output ANYTHING if called directly, and if you can cripple what they are allowed to do through dependencies, all the better!

    As you can tell, it's VERY deep and I'm trying to plug every PHP hole I can think of, instead of going the dipshit route found in systems like turdpress, where they put security info in DEFINE where it's global scope, undeletable until execution ends, and as such available anywhere and everywhere in the ENTIRE execution. (HERPAFREAKINGDERP!!!)

    Hey... you asked.
     
    Last edited: Apr 3, 2014
    deathshadow, Apr 3, 2014 IP
  15. deathshadow

    deathshadow Acclaimed Member

    Messages:
    9,510
    Likes Received:
    1,929
    Best Answers:
    247
    Trophy Points:
    515
    #35
    Oh, the laugh being that some of that CMS code even worked it's way into my poor man's... like "one index to rule them all", though it's using a more classical templating system and of course, being poor man's there's no DB, just static files strung together.
     
    deathshadow, Apr 3, 2014 IP
  16. deathshadow

    deathshadow Acclaimed Member

    Messages:
    9,510
    Likes Received:
    1,929
    Best Answers:
    247
    Trophy Points:
    515
    #36
    Just an update, almost done with the initial docs, though I've ended up rewriting some pretty big sections of it so I have to go through and proof the changes, as well as test the changes. The codebase is now officially Beta -- as in what Beta MEANS; feature complete but bug-ridden/untested -- as opposed to how people are throwing the word Beta around on things that aren't even feature complete... or worse full revision numbers/major versions (see the steaming pile of crippleware known as Opera 15 through 21)
     
    deathshadow, Apr 6, 2014 IP
    AbstractChaos likes this.
  17. deathshadow

    deathshadow Acclaimed Member

    Messages:
    9,510
    Likes Received:
    1,929
    Best Answers:
    247
    Trophy Points:
    515
    #37
    Oh, my "get" function for getting elements using CSS selectors was originally using my own parser -- I've added detection to use Element.querySelector if available, and found a polyfill that was... ok... but after a rewrite I sped it up 6x by walking the DOM instead of grabbing all elements and stuffing them in an array.

    Element.get and Element.getAll
    	get : (Element.querySelector) ? function(s) {
    		return _.extend(this.querySelector(s));
    	} : function(s) {
    		_.style.addRule(s, 'fu:bar');
    		var e;
    		if (e = this.nodeFirst()) do if (e.currentStyle.fu === 'bar') {
    			_.style.removeRule(0);
    			return _.extend(e);
    		} while (e = _.nodeWalkNext(e, this));
    		_.style.removeRule(0);
    		return null;
    	},
    
    	getAll : (Element.querySelectorAll) ? function(s) {
    		return _.extendList(this.querySelectorAll(s));
    	} : function(s) {
    		var r = [], e;
    		_.style.addRule(s, 'fu:bar');
    		if (e = this.nodeFirst()) do {
    			if (e.currentStyle.fu === 'bar') r.push(_.extend(e));
    		} while (e = nodeWalkNext(e, this));
    		_.style.removeRule(0);
    		return r.length>0 ? r : null;
    	},
    Code (markup):
    _.NodeWalkNext is fun too... familiar if you know my nodeWalking code.
    	nodeWalkNext : function(e, s, m) {
    		return e.nodeFirst(m) || e.nodeNext(m) || (
    			e.parentNode == s ? null : e.parentNode.nodeNext(m)
    		);
    	},
    Code (markup):
    e == element, s = stop element, m = mask

    Mask is used in alll the node walking functions (nodeFirst, nodeNext, nodePrev, NodeLast, NodeWalk, NodeWalkNext) -- if Mask is a string it's a tagName to be matched -- if it's a number it is a bitmask where each bit corresponds to a nodeType thus:

    Name                    NodeType    Mask      Mask Decimal
    ELEMENT_NODE                1       0x0001         1
    ATTRIBUTE_NODE              2       0x0002         2
    TEXT_NODE                   3       0x0004         4
    CDATA_SECTION_NODE          4       0x0008         8
    ENTITY_REFERENCE_NODE       5       0x0010        16
    ENTITY_NODE                 6       0x0020        32
    PROCESSING_INSTRUCTION_NODE 7       0x0040        64
    COMMENT_NODE                8       0x0080       128
    DOCUMENT_NODE               9       0x0100       256
    DOCUMENT_TYPE_NODE         10       0x0200       512
    DOCUMENT_FRAGMENT_NODE     11       0x0400      1024
    NOTATION_NODE              12       0x0800      2048
    Code (markup):
    Basically nodeMask == 1 << (nodeType - 1)

    So if you wanted to walk elements and textNodes only, you'd pass it a nodeMask of 5. (1 + 4)

    Orginally I had nodeWalk and tagWalk as separate functions, but figured that with so much of their code identical, it made more sense to make them single functions. I did the same with getting rid of textAdd and textReplace, combining them with nodeAdd and nodeReplace. If you pass a string, document.createTextNode is used, if you pass an object it is used directy.

    It's kind of an old-school Pascal trick (that doesn't work in modern flavors) -- where you could have functions with the same name but different parameter types treated as two separate functions despite the same name. Parameters were treated as part of the function name. With JS' lack of true typecasting, it's easy enough to detect the variable type and handle it appropriately.

    I'd prefer to have a dozen functions I can remember the names of that I might have to go to the reference for the exact syntax, than four dozen functions I'll never be able to remember all the names of.

    Probably why I call most of HTML 5's new tags "pointless bloat" and "redundant"...
     
    Last edited: Apr 6, 2014
    deathshadow, Apr 6, 2014 IP
  18. deathshadow

    deathshadow Acclaimed Member

    Messages:
    9,510
    Likes Received:
    1,929
    Best Answers:
    247
    Trophy Points:
    515
    #38
    Just a status update:
    closed Beta : complete
    base documentation : complete
    Internal testing : 60%
    Demo's: need compatibility rewrite

    Right now I'm writing out a whole slew of test cases to not just test that the functions work properly, but also that they FAIL properly. Had one fun slip-up, I wasn't set up to actually test legacy IE anymore since I built my newest workstation (i7 4770k, 32 gigs RAM), so I've got to get virtualBox set up there. )

    Fun how even the simplest routines can have mistakes in them... in several cases I forgot to actually return my return values (doh!)

    Right now it's sitting at 14.9k, though I can't think of much to add to the core file.

    I'm thinking on 'modules' to extend it further with -- I'm thinking my canvas wrapper might make a good one since it makes the canvas element from scripting (so who gives a **** what doctype you use it in -- NOT that there should even BE a CANVAS tag in the first damned place), creates a 2d context, and can auto-scale to fit the entire canvas as it does what CANVAS was MEANT to do -- separate pixel size from vector size. I think that's definitely going on the to-do list.

    Though the first demo is going to be an image rotator/slider, where the same JS can be used to trigger a whole slew of different CSS3 animations.
     
    deathshadow, Apr 8, 2014 IP
  19. deathshadow

    deathshadow Acclaimed Member

    Messages:
    9,510
    Likes Received:
    1,929
    Best Answers:
    247
    Trophy Points:
    515
    #39
    Status Update --

    I now have one and a half pages of documentation two write before it goes live -- that's out of 70 pages of content in total for the website...

    Love how a one week personal project turned into a month -- though admittedly I've only averaged 2 hours a week on it in my spare time. But as always, content is king, which is why I've held off so long. I could have tossed up a half-assed incomplete site weeks ago; but why make a half-assed implementation at the start?

    Giving myself an ultimatum this week -- I launch before friday, or I deep six the whole thing as a waste of time.
     
    Last edited: Apr 28, 2014
    deathshadow, Apr 28, 2014 IP
  20. AbstractChaos

    AbstractChaos Member

    Messages:
    58
    Likes Received:
    4
    Best Answers:
    4
    Trophy Points:
    25
    #40
    So did it get release or did you deep six (had to google that term) the project?
     
    AbstractChaos, May 6, 2014 IP