# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     language: python
#     name: python3
# ---

# # Employees performance and delivered value

# ![Employee performance](performance.jpg)

# ## 1. Description of the topic

# The aim of this project is to show the variables that can affect the employee performance in an IT company and the value delivered.
#
# The data sets will be simulated based as much as possible in the real world behavior and the variables will be studied to determine the ones that could cause more impact and to see tendencies when linking them.
#
# Based on experience, on personal research on the workplace and investigation on the topic, some of the factors that could influence performance are: Time/hours in the workplace, Initial level of studies, employee motivation, salary, training /conferences the employee attends, length of employment, experience in the position, number of meetings the employee has in a day, number of interruptions per day, emails received per day...
#
# In terms of the performance, the concepts related are: can be measured based on different approaches: ROI (Return of Investment), delivered value, velocity (measured in story points), incidents in production, defects per story point, customer satisfaction, time to market or completion of individual goals.
#
# This project will focus the simulation on the following 5 items:
#
# - Time/hours in the workplace
# - The initial level of studies
# - Employee Motivation
# - Salary
#
# The outcome would be presented as the Revenue or value that the company gets.

# **1.1 Investigation of existing research**
#
# - _Linking Belgian employee performancemanagement system characteristicswith performance management systemeffectiveness: exploring the mediatingrole of fairness_
#
#     Summary: The paper studies the effectiveness of Performance Management System, the variables that influence it and the outcomes of it. 
#     
#     The main conclusion is that the way of implementing the system has a big influenece in the outcome.
#
# https://www.researchgate.net/publication/46469879_Linking_Belgian_employee_performance_management_system_characteristics_with_performance_management_system_effectiveness_Exploring_the_mediating_role_of_fairness 
#         
#    
#
# - _Data: Hours worked in 2017_
#     https://data.oecd.org/emp/hours-worked.htm
#     
#     Summary: Data per country: mean hours worked in 2017
#     
#
# - _The Association of the 24 Hour Distribution of Time Spent in Physical Activity, Work, and Sleep with Emotional Exhaustion._
#
#     Summary: Interesting approach on how to pressent the results of the study with Ternary plot.
#     
#
# - _How do you measure value?_
#
#     https://www.thoughtworks.com/insights/blog/how-do-you-measure-value
#
#     Summary: Key factors and graphical representations on how to measure value in a company.
#     
#
# - _Infographic: You waste a lot of time at work_
#
#     https://www.atlassian.com/time-wasting-at-work-infographic
#
#     Summary: Factors that can reduce employee performance.
#     
#
# - _In an 8-Hour Day, the Average Worker Is Productive for This Many Hours_
#
#     https://thriveglobal.com/stories/in-an-8-hour-day-the-average-worker-is-productive-for-this-many-hours/
#
#     Summary: Article about how more hours at work could not translate into more productivity.
#     
#
# - _Wages and Employees Performance: The Quality of Work Life as Moderator._International Journal of Economics and Financial Issues, 2015, 5(Special Issue) 349-353.
#     
#     Summary: Research about how the effect of wages on employees performance are moderated by the quality of work life. It takes into consideration as well promotion policies, democratic supervision, employee involvement and safe working conditions, perception that they want to feel safe, satisfied and get a chance to be able to grow.
#
#     It presents the descriptive statistics about the data in a table.
#     
#
#
# - _Impact of Motivation on Employee Performances: A Case Study of Karmasangsthan Bank Limited, Bangladesh_
#
#     https://www.omicsonline.org/open-access/impact-of-motivation-on-employee-performances-a-case-study-of-karmasangsthan-bank-limited-bangladesh-.php?aid=86681
#
#     Summary: In this research salary is described as an extrinsic motivation along with Monetary Incentives and Compensation Package.
#
# ![Salary impact](salary.png)
#
#
# The representation above is extremely valuable for this project as it can be seen that the relationship between salary and motivation will be similar to a exponential distribution.
#
#
#
# - _Data Distributions_
#
#     Summary: Brief of data distributions.
#     
# http://www.ucd.ie/ecomodel/Resources/Sheet4_data_distributions_WebVersion.html
#
#
#     
# - _Education you need to work at companies like Facebook, Google and Amazon_
#
#     Summary: Level of studies distribution in top tech companies.
#     
# https://www.cnbc.com/2017/07/26/how-long-youll-need-to-go-to-school-to-work-at-top-tech-companies.html
#
#    
#

# ## 2. Values in the data set

# ### 2.1 Data Set simulation
#
# The dataset simulation will contain values of 1000 employees in a particular month. 
#
# Each parameter will be simulated according to the distribution more suitable for each of them based on experience and existing papers and research.
#
# For this project, we will take a month of work from the employees with different salaries, motivation, level of studies, hours at work and we will compare these variables with the revenue that the company received from that month. The ROI will be a fixed value as the income received because of this feature is a fixed figure. 
#
# For this simulation, the value obtained with this simulated feature will be 10.000.000. 
#

# ### 2.1.1 Time/hours in the workplace
#
# From the initial research, we could differentiate between time spent at the workplace and productive working hours. From a recent research, just 2 hours and 53 minutes of the time spent at the workplace is productive work time.
#
# _Source: https://thriveglobal.com/stories/in-an-8-hour-day-the-average-worker-is-productive-for-this-many-hours/_
#
#
# In this project, we will simulate the number of hours spent at the workplace as it is usually the measurement taken into account by most of the traditionally in most of the companies. 
#
#

# **_Hours at work data set Simulation_**

# %matplotlib inline

import pandas as pd
import seaborn as sns
import numpy as np
import matplotlib.pyplot as plt

#Mean Hours worked in Ireland 2017, 
#Source: The Organisation for Economic Co-operation and Development (OECD)
TotalhoursIreland = 1738
#calculation mean hours per day (weekends, bank holidays and Annual leave excluded)
#working days in Ireland 232 
meanhd= 1738/232
meanhd

#Working hours data set simulation
hours= pd.DataFrame ({"hours": np.random.normal (7.5, 0.5, 1000)})

hours

#small test to check that the data set fits the expected
np.mean (hours)

# The test gives a positive result as the number is very close to the defined mean for the Numpy Random normal function

# The working/hours dataset simulation has been generated using the normal distribution. We assume for this study that the company has the same working/hours policy for all employees. We assume that the policy is to work 7.5 h per day, as it's the mean of time spent working in Ireland in 2017. 
#
# _Why a normal distribution?_
#
# The distribution will follow a bell curve where most of the values will be close to 7.5. In the Normal or Gaussian distribution, the variables concentrate around a value and there are very few variables from 3 standard deviations.
#
# In this case, most of the employees would be working around 7,5 hours even though for this particular day that we are taking for the data some employees would stay a little bit longer at work or a little bit less than 7.5h.
#
# The mean for this random normal distribution is 7.5h, the standard deviation 0.5 as we assume that the employees follow the company policy and they don't change much the time they spend at work: 30 min is an acceptable deviation.
#
# As we expect the ROI of a feature to be obtained after a month of employees work, we will calculate as well the total of hours per month per employee.

#getting the hours per month 
hours_month= hours * 20
hours_month

# ### 2.1.2 Level of Studies

# From the research, in the table below we can see the percentage of employees of each level of studies in some of the biggest tech companies.

# ![Level of studies](studies.png)
#
# _Source: https://www.cnbc.com/2017/07/26/how-long-youll-need-to-go-to-school-to-work-at-top-tech-companies.html_

# **_Level of studies data set simulation_**

# +
#the distribution of studies level is actually distributed according to percentages
#based on data from Airbnb employees 14% undergrad, 53% bachellor, 21% masters, 12% phd
#https://docs.scipy.org/doc/numpy/reference/generated/numpy.random.choice.html#numpy.random.choice

undergraduates = 0
bachellors = 1
masters = 2
phd = 3
studies= pd.DataFrame ({"Studies": np.random.choice(4, 1000, p=[0.14, 0.53, 0.21, 0.12])})
# -

studies

#small test on the data
np.mean (studies)

# The test gives a positive result as the number is around 1, the parameter with more probability to appear defined in the np.random.choice funtcion.

# We assume that our company is an online company as Airbnb that highly relies on technology to operate their business.
#
# The np.random.choice function is used to generate a non-uniform random sample.
#
# Why np.random.choice?
#
# The function used to generate this random distribution is np.random.choice as we need to generate a random list that fits in the percentages above: based on data from Airbnb employees 14% undergrad, 53% bachelor, 21% masters, 12% Ph.D. In order to do that np.random.choice provides the ability to set the percentages of the different variables to occur.
#
# Thos figures are based on the investigation, of the real numbers of Airbnb that we consider acceptable for this project as an online company that offers services to final users.
#
#

# ### 2.1.3 Motivation
#
# What is motivation?
#
# Motivation is the word derived from the word ’motive’ which means needs, desires, wants or drives within the individuals. It implies Internal and external factors that stimulate desire and energy in people to be continually interested and committed to a job, role or subject, or to make an effort to attain a goal.
#
# Motivation results from the interaction of both conscious and unconscious factors such as the intensity of desire or need, incentive or reward value of the goal, and expectations of the individual and of his or her peers. These factors are the reasons one has for behaving a certain way.
#
# In the work goal context, the psychological factors stimulating the people’s behavior can be -
#
# - Desire for money
# - Success
# - Recognition
# - Job-satisfaction
# - Teamwork, etc
#
# _Source: https://www.managementstudyguide.com/what_is_motivation.htm , http://www.businessdictionary.com/definition/motivation.html_
#
#

# **_Motivation simulation_**

# +
#the motivation of the employees in a company would be totally specific to that company
#here the parameters for this company

No_motivated = 1
Somehow_motivated = 2
Motivated = 3
Very_Motivated = 4
Extremely_Motivated = 5

#as the case will be specific to the company we will calculate a totally random distribution
#Assuming that each value has the same probability of being choosen
#Use randint to generate random integers from low (inclusive) to high (exclusive)
#following the uniform distribution
motivation= pd.DataFrame ({"Motivation": np.random.randint(1,6, size=1000)})
# -

motivation

#small test on the data
np.mean (motivation)

# The test is sucessful as, using np.random.randint we get random integers from the “discrete uniform” distribution all the numbers have approximately the same possibility of appearing. Te number in the middle of this set of motivation numbers is 3, so if the mean approaches this value the data set performs correctly.
#
#
# _Why np.random.randint?_
# np.random.randint allows to define the interval, setting low and high, needed in this case to set the low to 1 and the high to 5. As the high is not included in np.random.randint, the high in this case is set to 6, so the numbers into consideration are 1 to 5.
#
# The uniform distribution has been chosen in this case and we don't have refence data we assume that any value has the same possibity of apprearing.
#
# We could as well have choosen a standard normal distribution (bell curve), centering most of the data to a value, assuming that most of the employees would feel that value of motivation, but as we don't have any reference of what this mean value would be, we assume that any employee can have the same posibility of being motivated or de-motivated.

# ### 2.1.4 Salary
#
# Salary or wages has been seen as one of the major components to incentivize employees performance, but is it really the main factor? What would be its data distribution? 
#
# Regarding the questions above, should the dataset simulate data on employees in similar positions but different companies to get the most of this project?
#
# https://www.econjournals.com/index.php/ijefi/article/download/1403/pdf
#
#

# **_Salary Simulation_**

# Taking as reference: https://www.indeed.com/cmp/Airbnb/salaries

#Mean of the salaries per employee per year
salaries = [157357, 138554, 172386, 164060, 176814, 133565, 138554, 139526, 116099, 122039, 196315]
SumSalaries = sum(salaries)
Meansalaries = SumSalaries/11
Meansalaries

#standard deviation
# analyse from the mean what is the deviation to the higher value
#higher value (high standar deviation hsd)
a = 196315
hsd= a - Meansalaries
hsd

#standard deviation
# analyse from the mean what is the deviation to the lowest value
#lowest value, lower standar deviation (losd)
b = 116099
losd= Meansalaries -b
losd

#mean standard deviation of the highest and the lowest
sumst=(hsd + losd)
#most of the values will be within 3 standard deviations
meanst= sumst/3
#result in average between the 2 values
meanstsfinal= meanst/2
meanstsfinal


#Using the standard normal distribution as is more probable to get a salary around 150479
#This mean wll be used as the mean for the standar normal distribution
salary= pd.DataFrame ({"Salary": np.random.normal (150479, meanstsfinal, 1000)})

salary

#small test on the data
np.mean (salary)


# The test is correct as the mean for this distribution is 150479 $. 

np.max (salary)

np.max (salary)
np.min (salary)

# In terms of the minimum and the maximum in the set, we can accept those figures as acceptable as they are close to the limits we have in the reference set.
#
#
#
# *A revision on how to set the standard deviation has been made as in the first attempt the standard deviation was taken just from the higher deviation. There was a rework on the standard deviation now obtained by the mean of the higher and the lower values.

# The salary is normally distributed around the mean, with a smaller probability of getting 196315 $ then to get the values around the mean 150479.
#
# _Why standard normal distribution?_ 
#
# As per the data observed most of the salaries vary in a range between 130000 dollars and 170000 dollars so getting the mean of all the salaries will set this intermediate point. The maximum 196315 dollars or the minimum 116099 dollars are rare exceptions, so setting the deviation in this range can give an approximate simulation of the salaries in a company.

#year salary divided in 12 months
month_salary= salary/12
month_salary

# ### 2.1.5 Revenue
#
# The revenue is presented as a fix value of 10000000. We assume for ths project that when the employees deiver the sotware, the income that the company receives in return is 10000000.

# ### 2.2 Concatenating the simulated data

# +
#Concatenating the simulated data above
dataset = pd.DataFrame({"Hours month":  np.random.normal (7.5, 0.5, 1000)*20 , "Studies": np.random.choice(4, 1000, p=[0.14, 0.53, 0.21, 0.12]), "Motivation": np.random.randint(1, 6, size=1000), "Salary": np.random.normal(150479, 15278, 1000)/12, "Company Revenue": 10000000})                       
                                                            
    
# -

dataset

# ### 2.3 Syntesizing the data set

#analysis of any data set 
dataset.describe()

# The description function presents very valuable information, as it shows for our data set:
# 1. COUNT: The number of values of the specified column. In this case 1000 of values in each column.
# 2. MEAN: The mean of the values of each column. In this case the mean of the 1000 values that we simulated for each column
# 3. STD: Shows the standard deviation between values
# 4. MIN: The minimum value in the column
# 5. 25th percentile. 25% of the values are less or equal to the number that appears for each column
# 6. 50th percentile (medium) 50% of the values are bigger and 50% of the values are smaller than that the number that show per each column.
# 7. 75th percentile. 75% of the values are lower than this number per each column
# 8. MAX: the maximum value in each column

#mean of each column
dataset.mean ()

#first rows of the data set
dataset.head()

#last rows of the data set
dataset.tail()

# ### 3 Relationships between variables

# Taking the project to a more advanced level, most of the variables would depend on some of the other ones in the data set. 
#
# - The level of studies and the salary will be related. The higher level of studies the more probability of having a higher salary based on the knowledge that the employee gives to the company. 
#
# - The hours at work and the motivation could be as well related in two opposite ways. 
#
#     1. The employees would be more motivated if they have more time to enjoy out of the office. So when they are at the office they are more engaged.
#        
#     2. The employees more motivated stay in the office more time because they enjoy a lot they work, so then they are motivated.
#     
#
# - The salary and the income will be closely related on calculating the ROI (return of investment) higher salaries would rate less on ROI as the salary will be part of the costs of the company in calculating the ROI (revenue- costs).
#
# - The Salary and motivation could be also related. There are studies (some quoted in the Investigation) that explain that salary could have an effect in the motivation as an extrinsic motivation among other factors. And other references detail the very little relationship between salary and motivation and show higher relationships between motivation and public recognition, training, company culture...

# **Data set without relationship between variables Hours/Motivation**

#subset of two columns of data to visualise them
dataset1=dataset [["Hours month","Motivation"]]
dataset1

# +
#Synthesing data and showing relations 
# plot data and regression model
# https://seaborn.pydata.org/generated/seaborn.lmplot.html

sns.lmplot (x="Hours month", y="Motivation", data=dataset1)

# +
#Scatterplot
#scatter plot with possibility of several semantic groupings
#https://seaborn.pydata.org/generated/seaborn.scatterplot.html


sns.set(style="ticks")

sns.pairplot(dataset1, hue="Motivation")
# -

# Above we can see a Scatter Plot and regression model plot.
#
# At the moment as we can see there is not the relationship between the data as it was randomly created without taking into account any relationship between variables. 
#
#
#
# _Terminology_
#
# The **regression model plot** fits regression models across conditional subsets of a dataset.
# _Source: https://seaborn.pydata.org/generated/seaborn.lmplot.html_
#
# The **Scatterplot** draws a plot with the possibility of several semantic groupings.
# In a Scatterplot, the relationship between x and y can be shown for different subsets of the data using different parameters. These parameters control what visual semantics are used to identify the different subsets.
# _Source: https://seaborn.pydata.org/generated/seaborn.scatterplot.html_
# _Source: https://seaborn.pydata.org/examples/scatterplot_matrix.html_
#
#

# **Data set with relationship between variables Hours/Motivation**

# In the data above the concatenation is just a mix of all the random sets simulated previously, with no patterns or stablished models. 
#
# Now we will be relating the data sets to be able to prove or refute an hipotesis. 
#
#
#
#
# _**Hipothesis**_
#
# The employees who spent more hours at work are less motivated than the employees who stay less hours at the office.

#Simulating a basic inverse relationship between time spent at work and motivation
h= np.random.normal (7.5, 0.5, 100)*20
m = 450/h
data = pd.DataFrame({'HoursM': h, 'Motivat': m})
data

# +
#Plotting the data to show if it exists a relationship between the data
#https://seaborn.pydata.org/generated/seaborn.scatterplot.html


sns.set(style="ticks")

sns.pairplot(data)
# -

#Plotting the data to show the data and the regression line of the data
sns.lmplot (x="HoursM", y="Motivat", data=data)

# This simulated data would allow us to validate the hipotesis; we can see a clear relationship in both in the regression plot and in the Scatter plot between the hours spent at work and the decrease in motivation.
#
# The more hours the employees spend at work the less motivation they have.

# ### 4. Extra relationships simulations

# #### 4.1 Salary based on level of studies

#Simulating salary in base of studies level
#more level of studies = more salary
studies= pd.DataFrame ({"Salary": np.random.choice(4, 100, p=[0.14, 0.53, 0.21, 0.12])+1})
salary= studies * 80000
salary

# In this case, we can appreciate that the higher the studies level, the higher the salary the employees receive. 
# The calculation is based on multiplying the level of studies randomly generated (random.choice) based on Airbnb distribution, in the previous sections, per a fixed amount 80000 euros. 
#
# So the higher the studies level the higher amount of salary the employees will receive.
#
# In this specific case, as the undergraduate parameter was set to 0, the salary values for those employees would be set to 0. But adding one to the random function solves the problem of obtaining results set to 0.
#
#

# +
#Plotting the data to show if it exists a relationship between the data
#https://seaborn.pydata.org/generated/seaborn.scatterplot.html


sns.set(style="ticks")

sns.pairplot(salary)
# -

# The Scatter plot above shows the amount of employees that would receive a salary range (0, 80000, 160000 and 240000) related to the lever of studies (undergraduate, bachellor, master, and ph.D)

# #### 4.3  ROI Return of investment or relationship between Salary and Revenue
#
# **What is understood as value?**
#
# Based on the context of an IT company, the value will be presented as ROI or Return On Investment.
#
# The ROI measures the gain or loss generated on an investment relative to the amount of money invested. ROI is usually expressed as a percentage and is typically used for financial decisions, to compare a company's profitability or to compare the efficiency of different investments.
#
# The return on investment formula is:
#
# ROI = (Net Profit / Cost of Investment) x 100
#
#
# _Source: https://www.thoughtworks.com/insights/blog/how-do-you-measure-value_ , https://investinganswers.com/financial-dictionary/technical-analysis/return-investment-roi-1100 and https://barnraisersllc.com/2017/03/surprisingly-simple-steps-calculate-roi/_
#
#  

# +
#ROI calculation= revenue-investment/investment 
#Relationship between the benefits and the salary

salary = pd.DataFrame({"ROI": np.random.normal(150479, 15278, 100)/12})
c= 10000000-salary
ROI= c/10000000
ROI
# -

# The ROI shows that the higher the salary of the employee the smaller the ROI for the company, as the revenue remains constant.

# ## 5. Conclussion 
#
# The use of Numpy is very efficient for simulating data sets attending to the different distributions of the data. In this project we can see it in the different distributions used:
# - The Normal distribution (numpy.random.normal) to simulate data that tends to a number, as the hours spent at work by the employees of a company tending to 7 and a half hours, setting a standard deviation of 0.5 hours. Or the salary of the employees with the mean value and the standard deviation.
# - Random.choice to simulate data according to different percentages, as the level of studies of the employees in a company.
# - Random.randint to create data of integers following a discrete uniform distribution, all the values has the same chance of being choosen, as the level of motivation of the employees in a company.
#
# Pandas package has been very useful to generate data sets, show them, and concatenate various sets of data. Moreover offers interesting functions to summarise data such as dataset.describe.
#
# Seaborn package is of great use to visualize the data and find patterns, relationships, and models. Scatterplots and Regression plots can be very useful to find relationships between data or lack of relationships between them. These relationships can be easier to identify visualizing the data rather than seeing it in a data set.
#
# The combination of these three packages offers a powerful engine for data analysis.

# #### Sources relationships between data and modeling 
# https://www.scipy-lectures.org/packages/statistics/index.html
#
# https://www.theanalysisfactor.com/7-guidelines-model-building/
#
# https://www.theanalysisfactor.com/13-steps-regression-anova/
#
# https://en.wikipedia.org/wiki/Statistical_model
#
# https://www.theanalysisfactor.com/assessing-the-fit-of-regression-models/
#
# https://www.kaggle.com/keiichiroumiyamoto/linear-model
#
#
# #### Sources used to investigate code errors
#
# https://stackoverflow.com/questions/17839973/constructing-pandas-dataframe-from-values-in-variables-gives-valueerror-if-usi
#
# https://softwareengineering.stackexchange.com/questions/238033/what-does-it-mean-when-data-is-scalar
#
# https://stackoverflow.com/questions/43054217/typeerror-dataframe-objects-are-mutable-thus-they-cannot-be-hashed-while-s
#
# https://stackoverflow.com/questions/17839973/constructing-pandas-dataframe-from-values-in-variables-gives-valueerror-if-usi
#
# https://docs.scipy.org/doc/numpy/reference/generated/numpy.linspace.html
#
# https://stackoverflow.com/questions/36921951/truth-value-of-a-series-is-ambiguous-use-a-empty-a-bool-a-item-a-any-o
#
# https://medium.com/dunder-data/selecting-subsets-of-data-in-pandas-6fcd0170be9c
#
# https://www.hackerearth.com/practice/machine-learning/data-manipulation-visualisation-r-python/tutorial-data-manipulation-numpy-pandas-python/tutorial/

StarcoderdataJupyterScriptsDedupFiltered