Regression / Classification problem type identification improvement

[sanketsarang]

Refers to file: https://github.com/blobcity/autoai/blob/main/blobcity/utils/ProblemType.py

target_length =len(np.unique(data))
            if data.dtype in ['int','float'] and target_length<=100: 
                return dict({'type':'Classification'})
            else: 
                return dict({'type':'Regression'})

The above code is not the best way to differentiate between regression and classification.

Change logic to cardinality off column against length of column. If cardinality of column is greater than or equal to 50% of length, then consider as Regression. If cardinality is less than 50% of length, then consider as Classification.

[aadityasinha-dotcom]

Can I work on this issue?

[Thilakraj1998]

Can I work on this issue?

Yeah Sure

[aadityasinha-dotcom]

Hey @sanketsarang can I use this logic which is given below to differentiate using cardinality:

for col in df:
    cardinality = len(pd.Index(df[col]).value_counts())
    print(df[col].name + ": " + str(cardinality))

[Thilakraj1998]

Hey @sanketsarang can I use this logic which is given below to differentiate using cardinality:

for col in df:
    cardinality = len(pd.Index(df[col]).value_counts())
    print(df[col].name + ": " + str(cardinality))

@aadityasinha-dotcom

you have to check only the cardinality of target/dependent column to identify whether it falls under classification or regression problem

[Cipher-unhsiV]

Hey @Thilakraj1998, Can you assign me this issue??

[Thilakraj1998]

Hey @Thilakraj1998, Can you assign me this issue??

@Cipher-unhsiV here you go

[Cipher-unhsiV]

Thanks dude!!!

[Cipher-unhsiV]

@Thilakraj1998 kindly check my PR.

[Cipher-unhsiV]

@Thilakraj1998 dude, kindly approve my running workflow!!!! It's been a long time...

[Thilakraj1998]

@Thilakraj1998 dude, kindly approve my running workflow!!!! It's been a long time...

@Cipher-unhsiV
merged the suggested change 👍