Performance Analysis: Fallout 4
Updated with fresh frame-rate testing in more challenging areas of the game.
UPDATE 13/5/15 11:40am: Since our initial analysis, we've put more time into Fallout 4, and we've been able to isolate a number of spots that really challenge the engine. In particular the Corvega factory is a rough spot, and PS4 takes the brunt of the hit once we reach its rooftops (down to 15fps). Both machines also suffer long drops to 20fps in certain interior battles. Time will tell just how far-reaching these trouble spots are throughout the game, particularly once more hours tally up. In the meantime, we'll update soon with a full Face-Off, taking a deeper dive into the specific ins and outs of each platform.
Original story: Four years on from its predecessor, Bethesda emerges from the vault with the hugely anticipated Fallout 4 - adding new technologies to its Creation Engine on PlayStation 4, Xbox One and PC. Equipped with an improved physically-based lighting model, it's confirmed the game's development process began with the team porting its earlier work to Xbox One first. However, optimising for both platforms has evidently proven tricky since, and on analysing the reality of each console's frame-rate at launch, the results are something of a mixed bag.
First up though, we can confirm PS4 and Xbox One render at a native 1080p resolution, with no compromise in either case. It gives us a sharp base image, and in presenting a post-apocalyptic world that sprawls out for virtual miles in each direction, it pays off hugely in keeping the image sharp on views across the wasteland. It's a surprise for Microsoft's machine too, with its previous history of cutting back on resolution in similar open world games. But here, both consoles give a matching, full HD presentation, also using a temporal anti-aliasing pass to minimise shimmer on panning shots.
We'll be giving a more thorough analysis of the game's visuals in our full Face-Off. But initial testing shows PS4 and Xbox One's core graphics settings are surprisingly close across the board. Texture maps are matched for resolution, with a generous level of anisotropic filtering across the ground for good measure. Each uses the same grade of screen-space ambient occlusion, to match PC's highest, and effects quality is identical too. With everything being so close in the visual stakes though, how does the frame-rate on these machines hold up?
It's worth stressing at this point that Fallout 4 is, of course, a huge game with many avenues to explore - but out testing so far paints an interesting picture of each console's general level of performance. To start, the game targets a 30fps update here, with v-sync engaged. As Bethesda's console projects go, this is par for course; with the detail rich apocalyptic environments the game is gunning for, it makes sense to budget modestly to factor in all the variables at play. Objects, shadows and characters stream in at an identical range between PS4 and Xbox One, with a shifting day-night cycle and weather system as well. However, it's far from a perfect situation on consoles as far as frame-rate consistency goes, and neither can truly latch on to a rock-solid 30fps reading.
All things put into perspective, Fallout 4 does hold 30fps as a general rule, but drops are noticeable. For example, moving between major city areas is a particular strain on the engine, causing PS4 and Xbox One to drop to 20fps in matching spots on the map. Like clockwork, each platform lurches downward for a spell when passing a threshold in the environment - suggesting assets are being decompressed on the fly for the next location. 30fps is restored relatively quickly on arriving at the next spot, and it's business as usual from there. However it's not an ideal setup when these connecting areas are filled with enemies, and controller response suddenly takes a hit - though PS4 holds a frame-rate advantage in matching runs.
But that's not the whole story. On top of this, Xbox One is unique in its suffering of a stuttering issue, halting the game experience for up to a second during play. It's a glaring hitch downward, and matching runs to the gates of Diamond City shows Xbox One dropping to a record 0fps (zero) while PS4 turns the same corner at 28fps. Each has their blips, but having tested two separate Xbox One and PS4 consoles, the results are always the same across the world at large; we get sizeable stutters on Microsoft's console that aren't present on PS4.
It may only be momentary, but it sticks out like a sore thumb in direct comparison. The cause is also often very self-evident; even with an installation size of 28GB, assets are struggling to stream from Xbox One's HDD on cue. We see noticeable hitches when simply walking around the world - but also just before cut-scenes take place, or switching to a new gun via the d-pad. During battle, reaching for a weapon that hasn't been used for a long spell prompts anything from a split-second hitch to a prolonged hiccup that can really disrupt the flow of a shootout. Added to that, the stutter from a newly drawn weapon (or throw of a Molotov cocktail) often coincides with a delayed sound effect in its first use.
On repeat tests we were taken aback by disparity we saw. Even on multiple re-tests, using the same enemies, weather and lighting conditions, each Xbox One console we try has similar, and obvious issues streaming in data effectively - where it doesn't manifest to nearly the same extent on PS4.
It's worth stressing this only affects a weapon's initial use on Xbox One, and not in every single case. And once loaded, hot-swapping between weapons from there onwards is as seamless as you'd expect. However, it is a distraction, and inevitably, if a weapon is left holstered for long enough, drawing it again cues another stutter down the line. This streaming issue is far from a game-breaker on Xbox One, but it is an obvious blemish that isn't resolved on patch 1.01, especially in combat where a lurch in motion can have a big impact on aiming.
In complex GPU-bound tests, such as a walk around the bustling Diamond City, we also see a divide in performance between the two consoles. Though most locations hold mostly steady at 30fps, this is one of the more densely packed, detailed spots in the game in terms of overall NPC count and geometry layout. As a result, both PS4 and Xbox One are prone to drops when walking around its centre, and Sony's machine consistently gives us the smoother return. On overlooking the city as a whole, Xbox One lags behind at a sustained 25fps in this case, while PS4 delivers at 28fps until we leave the scene. There's no question in this sense that PS4 leverages its raw hardware advantage into a performance lead - despite resolutions and assets being identical with Microsoft's platform.
It's a clear result across city areas, but surprisingly PS4 has a shortcoming elsewhere. While most battles produce a similar overall frame-rate, shootouts involving heavy transparency effects can affect PS4 more negatively than we'd expect. In extreme cases, such as on approaching an early Deathclaw enemy in Concord town, PS4 wavers close to the 20fps line when letting rip on a minigun. Fire, smoke, and blood splatter effects collide, and on re-testing this section several times on both consoles, Xbox One consistently produces a near-unshakable 30fps. It's possible this is a side-effect of Bethesda's choice to begin engine optimisations on Xbox One - but even so, PS4's lengthy stretches at the low 20fps range are out of place, given its advantages in every other area.
Indeed, just about every other battle produces similar results between the two consoles. Tackling a zombie swarm in a later city area, for example, PS4's frame-rate doesn't struggle noticeably when compared to Xbox One - and drops appear from both sides. It suggests it's only a problem in extreme cases, where effects are used to excess. On balance though, PS4 is in the better shape as terms of consistency across the game; it has nothing close to the stuttering seen on Xbox One, and a better frame-rate in big towns and cities as well. Xbox One has a repeatable advantage in the Deathclaw battle, where effects clash heavily, but in most scenarios we get matching, or superior results on PS4.
But what about Fallout 4 on PC? Games based on Bethesda's Creation Engine have historically fared better on the PC platform, and our initial tests suggest that this trend continues with Fallout 4. We've been playing the game on our budget gaming PC, pairing a Core i3 4130 with a GTX 750 Ti and 8GB of DDR3. We used the game's high preset as a base, then dropped shadow quality and distance down to medium, with lighting pared back to high - this gives us a visual experience on par or better with console. We have enough horsepower left over to put its object fade setting on par with actor fade (in excess of the high setting) and the end result is improved performance and marginally superior visuals, still at 1080p.
What we lack is a decent 30fps cap, which sees performance lurch - Nvidia's half-rate adaptive v-sync produces ungodly stutter here, and it's down to Riva Tuner Statistics Server's frame-rate cap to level performance, though frame-pacing is still an issue. Even so, this is a great experience overall on cheap hardware, boding well for those with quad-core processors and better GPUs. As an aside, we can say that even ultra level textures still fit comfortably within a 2GB framebuffer at 1080p. VRAM utilisation isn't a problem for Fallout 4.
We'll be bringing a more detailed analysis later in our full Face-Off, with a focus on how console visuals stack up to PC's ultra settings. But as far as console performance goes with patch 1.01 installed to each, it's clear from our initial tests PS4 and Xbox One have issues holding 30fps - and Xbox One's stuttering issues stand out as a particular distraction. It begs the question of whether a faster drive, perhaps even an SSD, could remove Xbox One's hiccups when streaming in these new assets - something we'll look into as soon as we can.
For now though, evidence so far suggests those buying Fallout 4 on console should get an overall smoother experience from PS4. It's far from perfect, and we have a suspicion re-tests will be due once each machine receives new patches down the line. As ever, we'll update as new information comes in. Meanwhile, for those with PC as their platform of choice, all the signs look good for a smooth experience even on budget kit. We're currently putting together an in-depth guide to settings tweaks and recommended setups, so expect that in the next couple of days.
Additional reporting by Richard Leadbetter.
If you've come unstuck in the game, our Fallout 4 guide is live now.