Schema scheming

[AMmayhem]

Alright, lets quickly move the review schema to a new thread as Mass has suggested. I'll post some of the comments already posted that bring weight to the discussion. Plus I won't put them in the quote format, so it's easier to quote from, if that makes sense.

 

[AMmayhem]

Posted by Hourences:

maybe like giving him 5 fields to type in, 1 freestyle field, and 3 others are meant for talking about visuals, technical and gameplay
so its like


Review : omgblabla


Conclusions
Visual : OMG LIGHT SUX AND TOO MUCH BLUE AROUND SO THIS DRAGS DOWN MAIN SCORE
Tech : OPMG ALL WALLS ARE NON SOLID
Gameplay: BlA

Overall : Great map for all low grav insta players ! specially the unique technical things make this map stand out !


so that reviewer HAS to write a short text about all 3, just to sum it up, freestyle field can still hold anything else he wants like it is atm
just an idea

 

[AMmayhem]

Another post by Hourences:


3 numbers as bot said is way to confusing
say youre a noob
you open nalicity, see a number and within 0.1 sec you must understand whats the deal with the map, without first having to read some faq on how to read it
people leave then

the names really are overly complicated too, and certainly for non native english speakers, cast wtf, awe wtf, only build is more or less understandable
i still have no utter idea whats the real meaning behind the word "awe", i know its suppose to represent visuals on nc, but further then that i have no idea what kind of word it actualy is

when i review ironblayde i play map and i just come up with a number, it pops up in my mind in big giant flashying red neon light, 6 6 6 6 6 6, so i give map a 6 and then go see "how can i divide this 6 so it fits the awe cast and build score, oh lets do 1 for that, 2 for this, and then we have erm, 3 left so lets score other one 3"
that doesnt work
you feel what quality a map is like, you dont start ut and say "oh awesome this is a 2/3 build map and a 1.5/3 awe map", no you feel the general score
anything else is not natural

when this system was chosen there was a discussion about pleasing 2 groups of people, on 1 hand the group who like everything in numbers and very precise, and on other hand the people who want to know if a map is good or not but dont care at all about numbers, the group who wants a happy smiley face and a 2 line conclusion saying what the map is like
a single number review is the middle of these 2 groups, it is rightly in between the 2 opposites and has an element of both sides

 

[AMmayhem]

Posted by Shadowlurker:


i like the idea of havng three different sections but the sections themselves suck and confuse

Should just be
Visuals-
Gameplay-
Technical-

rather than
Awe
Cast
Build

because there are things in cast that link directly with things in build.
Make more sense to just call them by what they really are, IMO.

 

[AMmayhem]

Posted by Ironblayde:


Bot: True, but the individual category scores are already listed at the top of every review, and users have the ability to search for maps using category scores as search criteria... so what's the harm in having the final score listed as well? There should be some measure of a map's overall appeal.

Hour: Actually, I do come up with category scores in my mind before worrying about the final score. Nine times out of ten, when I add them up, I look at the overall score and find that it sums up my thoughts nicely. Once in awhile I'll end up with a final score that strikes me as a little high or low, and I'll go back and look at the categories more closely to see if I was perhaps evaluating something incorrectly, but for the most part the category approach works for me. It's all a matter of preference, I suppose.

Also, I know that a lot of people who visit the site haven't read the review schema, and go by the final score alone when checking out our reviews, but eliminating the categories isn't going to make things any easier for anyone. The final score is already there, and it's already representative of a reviewer's overall opinion, so what's the problem? If someone doesn't care about how the score is broken down, let him just ignore that part of the review. Someone else will care about all the details, though, and so it's good to have the additional information at hand.

 

[AMmayhem]

Posted by Bot_40:


I was a fan of keeping scores completely seperate, and calling them something more obvious like Envioronment, Gameplay and Technical. Then a score just shows up as (5,7,7) or something like that so at a glance you could see if a map is good gameplay wise of good on visuals etc.
Of course I guess that would be pretty impossible to impliment now I guess


And later:


If there is an overall score max of 10 then you also now have the problem of how you split that into 3. Might be better to have all 3 scores out of 10 then just use the average like NC2 but with less catagories?

 

[Nahand]

... erhm... i'm a little confused but... apart from the renaming of the criteria names (and subsequential switches of some categories to others, like pointed previously), the current system seems just fine. (Hourences)

If the problem are 1 line reviews... that's about the reviewer i guess. I won't do 1 line reviews, but i still refuse to spend more time in a review of a map that the author did building it. That doesn't mean i'll completely skip some details. Check CTF-B3 map review (still up, i think) at review list. I did inform in comic-relief style (if you want), but the details that made it suck were explained.

My opinion...

 

[MassChAoS]

I'd like to restate that renaming the current categories is really easy.

 

[AMmayhem]

For me, I tend to review things like Ironblayde. I look at the awe, build, and cast, individually, and the score comes out to about what I think the overall should be. There are exceptions though, (CTF-Orodruin for one), ones that just seem to deserve a better (or worse) score than what's coming up with the 3 categories; so I can understand where Hourences is coming from. But just deciding on one number overall with nothing else seems too random. Users may find it difficult to figure out where the reviewer "came up" with the number, but I suppose that's what would be discussed in the review.

I don't like the idea of forcing a reviewer to type the review in a certain way. There are different styles used by us reviewers. Qualthwar has always kept his short and sweet. I type my reviews in more of a format style, clearly separating the categories, and Ironblayde does similar as well. While Shadowlurker and Hourences tend to just discuss the map without the formalities that I do, but still keep things flowing.

So where's the middle road at this point? Damned if I know right now....


As for Bot's suggestion of having the categorical scores separate and visible. They already are at the top of each review. You can also search by the categories themselves. I do think it would be better though when searching this way, that you could see the score of the category(s) that you searched by, but I'm not sure there's enough room to add that. We could eliminate the combined user and review score, seeing that those two are already seen, and I think the user can figure the difference between them.

 

[Sarevok]

I'm with Hourences on this one, there's no need for a score for each catagory
just give the overall score on the map and you can just discuss the other categories in the review like gameplay, visuals etc.
we're making things to technical and mathematical here, let's keep things simple
also as I said in the other thread we should get rid of the 9 point scale and go back to the traditional 10 point one, doesn't make much sense for the reviewer to rate on a 9 point scale while the people who comment rate on a 10 point one
and finally we should get rid of the final point being dependent on the comments scores, the system is to abused with high scores to have any credibility, the final score should be up to the reviewer and that's it
http://nalicity.beyondunreal.com/map_hub.php?mid=311
just look at the scores here, there's such a huge gap between them, then you have assclowns throwing out 10's to anything so the map gets the extra point to put it at 2

 

[darth_weasel]

yeah sometimes one part of a map is good enough that it makes another irrelevant, example: dm-lego looks like **** but its gameplay is so good its not fair it only counts as 3 marks

im with leaving it down to the reviewer to discuss his own breakdown of the score in the review, taking it to a /10 scale, but not allowing reviewers to gives 10's (which they wouldnt/shouldnt anyways), so 9.5 would be the highest score, but dont score it like 9.5 is the highest, so we'd see more 9.5's than we currently see 9's, yknow? at the moment it seems since 9 is the highest score, you can think 'well this map isnt quite perfect so ill give it 8.5', even though the 9 is there for 'not quite perfect' maps.

 

[Chrysaor]

I think renaming them would help immensely, I don't get the names still. And I speak the language fine.

"Architecture is the masterly, correct, and magnificent play of masses brought together in light." Le Corbusier

I wrote a 12 page paper on how level design is architecture, and architecture is usually split into 3 categories revolving similarly around what you have divided now. I like when reviews talk about the individual categories, but they do interact to give that general impression.

I'm sure you guys will figure it out.

 

[Twrecks]

1st off Hourences is ghey. Scoring maps as Art is absurd, remove any objectivity and be dispositioned to "suXXorz" and "OMFG this map Rulez!".
I'm the first to admit the naming is "gimmicky", as was meant to be. Review previous discussions on the schema as to its origins. As to what "Awe", "Build" and "Cast" mean? FFS click a link from time to time, it's spelled out and the root meanings of the headings are about as confusing to non-english speaking ppl as any other words you might like to choose, unless we code the site to display multiple languages... NOT.

The 3 catagory system was developed as a medium between the 10 catagory each with 1-10 range each weighed differently AND the 1 score for all as suggested by the "Art" ppl. Only a MAX score of 9 because NC was deemed idiots for map reviews like Scimitar. We even have a review approval system for maps that score 7 or better BTW, Maz and Yoda would need atleast 1 more admin/reviewer to agree before posting trite like that ever again (of course we were new and all that BS)

The purpose of the 3 catagory system was also to force reviewers into justifying their scores. Sure, no matter what it's opinion based on reviewer experience/knowledge. But that's got to be better than some neon sign from God flashing a score, one would think anyways.

Next was S I M P L I C I T Y .
Adding 3 scores together I know is rather taxing. Reading a review to find out how that score was derived is even worse. Shoot, you wouldn't want anyone to actually visit our site or take interest in the drivel written here. Nexus files had no written reviews, just a number attached to each map, oh that roXXorz, not even a frickin screenshot LOL!

As to AMmayhems/Bot40 last comments on seperation, the UMP (or Unversal Map Profile) was abondoned early on. I mean if ppl get confused so easily now, just think if you gave a map a 627? (that's 6 for Awe, 2 for Build and 7 for cast), that's a 7.5 average right? You would need Jesus to tell you that every time.

To quote Michael Jackson, the schema now "is as easy as A B C, 1 2 3" I'm not saying that because I developed it, I mearly condensed what had been said in the past and came to a logical compromise. Really, if you guys want to dig up the past, search the forums first, previous discusions were far more detailed and constructive.

 

[Hourences]

oh oh oh i pissed someone off
oh wait, its the same person who was also already pissed at me 3 year ago when the review scheme got created i said it was bad
loosing calmed is weak

Scoring maps as Art is absurd, remove any objectivity and be dispositioned to "suXXorz" and "OMFG this map Rulez!".

-a. mapping is art, and if something is art then it should be reviewed like art
in fact i wrote a 2 page article about this for vandora full of arguments and thoughts, you are thinking of art in the wrong way
if mapping wouldnt be art, then any dickwit could make good maps
if you think its not then ill be happily awaiting a few pages of arguments and thoughts about the subject
-b. please explain what suxoromfhg glrules has to do with mapping as art, i really do not understand the connection between the two

As to what "Awe", "Build" and "Cast" mean? FFS click a link from time to time, it's spelled out and the root meanings of the headings are about as confusing

-so you are saying, if a first timer visits NC he should go take a dictionary or search the site just to understand some of the basics the site offers
thats not userfriendly. that would be like making a map where you spawn in a small room with a hidden exit and let the guy search an hour for the exit
he wont play that long, the names are just very not straight forward
-multiple languages has nothing to do with this, creating unneeded steps to further confuse potential users does
one of the base rules when creating user interfaces: make it dumb simple

for all as suggested by the "Art" ppl

who are the art people, apart from "the people who like art" and why are they art people, any definition for this ? never heard of the term before

Only a MAX score of 9 because NC was deemed idiots for map reviews like Scimitar.

now this was a very noble idea, and a good one but imo at the end it doesnt work out
instead helping and balancing the scores, by not giving out any of those unrealistic 10's it does the invert
you dont think anymore like "oh this map is 8/10", you start to think "oh this map is 8/10 but i only have 9 points so its 7.5/9" which the end users see as 7.5/10
now i dont know about you but to me a 7.5 is "good", while an 8 is really good. and the map would have been 8, deserves 8 but only gets 7.5 at the end, and that is confusing to the users
for some maps it helps downgrading the score, so the scores are more realistic, not overrated but for other maps it just messes it up
at the end your reviewers need to be competent, thats all, you cant save yourself with rules like these
no reviewer would give a map a 10/10 while it sux, and if he does he a. should get fired, b. wouldnt get trough the damn appoval anyway
and if he rates a 9.5 map a 10 then so what, that 0.5 point..
+ i think all of the current reviewers are smart and realistic enough to never give out a 10

The purpose of the 3 catagory system was also to force reviewers into justifying their scores. Sure, no matter what it's opinion based on reviewer experience/knowledge. But that's got to be better than some neon sign from God flashing a score, one would think anyways

some people do like this yes, but on other hand some people dont...
at the end you are using maths to calculate a score, it stands in the way of the simple thought "is this map good"
you use 1 group of people as argument, i could user the other..
+ everyone i know whos good can play a map or even just look at screens and say a number, that number just popped in his head and most of the times those numbers are spot on
where exactly is the problem with that ?

Adding 3 scores together I know is rather taxing. Reading a review to find out how that score was derived is even worse. Shoot, you wouldn't want anyone to actually visit our site or take interest in the drivel written here. Nexus files had no written reviews, just a number attached to each map, oh that roXXorz, not even a frickin screenshot LOL!

its about the idea behind it that you use maths to calculate a map, that is absolutely not possible and against and logic and nature
this is not science, this is art and design, you dont use maths .
if one cant make a score without doing maths he sucks in judging maps
and dont overstate things, thats a weak way out, i could do that too

you know i saw a pigeon today, and it like flew trough the sky, and i also saw an oceancruiser and it was like far bigger then well an ocean cruiser and then this woman came and she trew a brick at the ship and then the ship sunk and this giant wave lifted a monstertruck in the air and like slammed it on the woman

see, thats interesting

I'm not saying that because I developed it,

looking at your text you take it quite personally tho

previous discusions were far more detailed and constructive.

hm, i was told to **** off and that all my ideas suck a lot of goats i remember, but if thats what you call constructive then be my guest
I also seem to remember that half the people who made that system now genuily hate nc, wow great

 

[AMmayhem]

-Our scoring system is already a 10 point spectrum. 0-9 Technically 20 points if you count the halves. What more do you want? And how many maps are going to get that perfect score?

-Mapping is art. Yes, I can see what you're saying. But there's also "artists" in the art world that just dribble paint onto paper and somehow get paid millions for it. I don't know about other reviewers, but I'm not going to review maps like that.

Our scoring system is fine IMO, we don't need to change it. I've got more to say, but I've got to got to work. I'll post more later....

 

[Nahand]

... although i follow the review schema, that does not stop me to believe each and every map is indeed a work of art, in a way or another. Of course there's much "artist apprentices" out there ...

 

[Hourences]

-i would review such map, and i would score them 1/10 because from a visual technical point of view, it is not good
i dont see why not review those, just score them like they deserve

its very small minded to directly associate art with such paintings, art is not about paintings, its about bringing over a level of creativity and emotion for example, and its a very wide thing

art is also for a large part not opinion and a lot of art is strongly binded to lots of definitions and rules, ofcourse some art isnt but that are exceptions
lets take tradional art as the example atm

the scoring system has a 9 scale
3*x = 9/x
if you keep the 9 then make it clear to the people that we use a different scale then they are used too, put a big /9 next to the score for example
and then what do you get ? then you get dude nr 5478721 who reads a review and reads this 7/9, and he then will go like "hmm cool this map got 7/9, but how much would that be at 10, hmh hmh hmhmhmhmh ah it would be 7.5-7.75 !, so 7.5 or hm 8 cause 7.75 is near 8, tho good map !"

so he needs to think more before hes fully able to understand the score correctly

and if you simply dont put the 9 behind and give the people the idea that its /10 you get some maps who are scored too low...

it proves the system is not waterproof
again changing it right now would also cause a lot of probs so thats not gonna help alot either..

 

[Twrecks]

No, I'm still a retard and you're still ghey H.

12 pages condensed: Mapping is NOT an art, it's a CRAFT. Craftsmen must balance the available tools and the intended use to derive a final product. Maps are a product, and any asshat can make a map and get paid for it, hence they must be "good" right?
Art is made solely for appreciation, unless you are making a flyby, most maps are interactive and by gametype imply purpose.

At NC we are attempting to qualify a score, in regards to "art" scores have no place. However inquiring minds want to know HOW their map stacks up, so we try to answer that question. If they need a dictionary to read the review than maybe they will also learn something from the experience. If they don't want to read anything we still post a single number as a review rating, can we get any simpler?

Art ppl? They know who they are. Arrogant bastards who have no regard for value in "sub-standard" work. Standards that are often over inflated and subject to wims. nuff said.

Scoring was kept to 9 because the new NC wanted to change its image. I don't really care if it went to 10 again. If anything give the reviewer an extra +1 catagory for Art's sake.

Perhaps we are trying to hard to make map scoring a science? uh, NO. The schema is pretty lax. Weighing every aspect of a map is daunting and is subject to varied opinions. If a better solution exists I'm game. Some math must be used if their is to be a score, even a single score, the reviewer must asess the whole by a sum of its parts and compare them in a common scale. It must be possible or their would be no review sites. Numbers popping into your head or anyone else's is based on that same criteria, subconcious or conciously. A reviewer must/should qualify that score, it's their job really. We allow user comments, and sometimes those scores there are justified... sometimes.

"you know i saw a pigeon today, and it like flew trough the sky, and i also saw an oceancruiser and it was like far bigger then well an ocean cruiser and then this woman came and she trew a brick at the ship and then the ship sunk and this giant wave lifted a monstertruck in the air and like slammed it on the woman"

WTF? Somebody just exercised their 1st admendment rights.


EDIT:
And the score is x/10, user comments fill the gap.

 

[Hourences]

ah calmer already, weird, everytime i say that the next post always is a lot calmer

No, I'm still a retard and you're still ghey H.

explain why im ghey
cause i dont follow your points ?
any further explanation, arguments etc ?
and not just your opinion ?

12 pages condensed: Mapping is NOT an art, it's a CRAFT. Craftsmen must balance the available tools and the intended use to derive a final product. Maps are a product, and any asshat can make a map and get paid for it, hence they must be "good" right?
Art is made solely for appreciation, unless you are making a flyby, most maps are interactive and by gametype imply purpose.

ok so what if its not art, do you care to explain to me then what for example compositing light colors is about ?
thats surely art ? art is about colors composition etc
thats what you do when making a map, that is art
someone who doesnt understand all that can make a map yes, but it will look terrible so i dont know if you could qualify that as a real map
its like building a little shed with wood, and then saying im a woodworker or whatever english word is

no youre not then, youre just a guy who made a shed thats about to collaps because of technical mistakes
you wouldnt call that a shed either

this is the same

no art, no map, thats nothing to discuss about that is a fact

i know people who know ued to the max, 100 percent, but who still cant make good maps because they lack the art aspect, they dont feel nor see what is good and whats not

At NC we are attempting to qualify a score, in regards to "art" scores have no place. However inquiring minds want to know HOW their map stacks up, so we try to answer that question. If they need a dictionary to read the review than maybe they will also learn something from the experience. If they don't want to read anything we still post a single number as a review rating, can we get any simpler?

you still have not explained yet from what moment on something can be defined as art scores
i score everything like i described above, so in that regard all my reviews are art scores, thank you
it also was about simple words basics words that are needed to understand the site, and not the review text itself, dont twist things around, it makes people believe less in you

Art ppl? They know who they are. Arrogant bastards who have no regard for value in "sub-standard" work. Standards that are often over inflated and subject to wims. nuff said.
[\quote]
no you start such thing, now you finish it
i
-a. want names
-b. an explanation why
-c. i want to know WHAT exactly art has to do with being arrogant, that must be one of the most pathetic attempts ive ever heard
what the hell in godsname does art have to do with that

Perhaps we are trying to hard to make map scoring a science? uh, NO. The schema is pretty lax. Weighing every aspect of a map is daunting and is subject to varied opinions. If a better solution exists I'm game. Some math must be used if their is to be a score, even a single score, the reviewer must asess the whole by a sum of its parts and compare them in a common scale. It must be possible or their would be no review sites. Numbers popping into your head or anyone else's is based on that same criteria, subconcious or conciously. A reviewer must/should qualify that score, it's their job really. We allow user comments, and sometimes those scores there are justified... sometimes.

is it lax ? yes it actually is pretty lax, did i say it wasnt ? i said it could use some improvent, whats wrong with that maybe ? maybe youre overstating quite a lot ?
if you call8/10 maths then yes, else i dont see why you need maths for getting a single score
oh so this number that pops in my head is based on that same principal ? thought it was a number from god that shouldnt be believed
but what if im catholic tho ? then i should follow the number god gives me, so imo we should make a religious NC

WTF? Somebody just exercised their 1st admendment rights.

wtf are amamdanements, its britisch flemish insane humor, used a lot here
its rather amusing even and in this case it served as example

the conclusion of having read your post makes me conclude several things
-youre pissed off by the fact i touch your thing
-you have a strong pre formed image and project everything that happpens on that image while you lack a lot of info
-therefor highly subjective and opinion based

i also feel a lot of anger in your post and possible frustration

(im a shrink btw)

i dont see why i should listen to something if those things were formed by above 4points, i dont see how it can ever come to a decent discussion then
you basically flame me for breaking down everything and being arrogant while im sorry to say but youre the one breaking me down now, and youre the one posting heavy replies stating youre right and all my ideas suck a lot...

you must chill and let the wisdom come to you
thats healthy for you


edit:
"And the score is x/10, user comments fill the gap"

give me 1 single example of where this system actualy worked !
wait, there isnt one..

 

[Bot_40]

hmm, first I don't like user comments swaying the score around, you may as well just have some code that decides whether a map gets +1 or -1 on the review score completely randomly given the amount of sensible, critical comments
Even if 100% of the comments were sensible, I don't see this system fully working. Say a reviewer rates a map 7/9 but everyone think's it's worth an 8, so everyone rates it an 8 but the review rating still never goes up because it has to differ by two. If the overall score was to change then everyone has to rate 9/10 which just messes everything up.

And I just don't like the whole scoring being out of 9 anyway simply because it's just natural to think in terms of out of 10. If you see a map with a score of 5 most people wouldn't even bother to look at it when actually it's above average. Same goes when you ask for a review of a really good map expecting 8/10 and suddenly it get's labeled "7".

I think the Cast Build and Awe are more important issues, they aren't self explanitory, and I did have to read the schema just to see what the hell it was supposed to be on about. I just think if it was Environment Gameplay Technical then it's completely self explanitory and nobody has any problems. I'm not saying completely change the way ratings are done or anything drastic, all it is is changing 3 names so first time visitors can actually understand it.
And don't just say "Go read the schema if you don't understand it", nobody who just visited the site for the first time, or even prolly the 20th time is going to bother to read the schema, they're just more likely to go elsewhere. And they certainly aren't likely to find it when it's 10 miles at the bottom of the page in size 0.1 font.